Developing and testing a theory

The companion vignette covers building a theory and checking its rigour. This article continues the workflow into development and testing. It shows how to operationalise severity, export a preregistration, appraise an amendment in Lakatosian terms, compile a measurement model for lavaan, and assemble a reviewer-facing audit bundle. The worked example reads the panic-network fixture shipped with the package, so every chunk runs with only theoryforge loaded.

theory <- tf_read(
  system.file("fixtures/panic-network.theory.yaml", package = "theoryforge")
)
theory$id
#> [1] "panic-network-2026"

Operationalised severity

A prediction earns evidential weight in proportion to the risk it takes. A point prediction stakes more on the data than a directional one, and that asymmetry should be made explicit before any test is run. tf_severity() reports, for each prediction in file order, the riskiness of its claim form and the resulting computed severity.

tf_severity(theory)
#>   prediction_id        type risk_score computed_severity
#> 1         pred1       point        0.9               1.0
#> 2         pred2    interval        0.7               0.7
#> 3         pred3 directional        0.4               0.3

The risk_score reflects the prediction type, while computed_severity applies the discounts and bonuses defined in the specification. Reading these values before data collection guards against the temptation to read a weak, hard-to-fail prediction as though it were a stringent test.

Preregistration

tf_preregister() renders a preregistration document as a single markdown string. It records the predictions, each with its claim type and derivation, plus the severity values, in a fixed order, so the output is stable across runs and suitable for deposit. Because the string is markdown, it renders below as the document a registry would receive.

prereg <- tf_preregister(theory)
cat(prereg)

Preregistration: Network theory of panic disorder

• Theory ID: panic-network-2026
• Schema version: 1.0
• Maturity: developing
• Derivation chain verified: yes

Hypotheses

1. [point] An interoceptive challenge raises arousal to a specified level within 90 seconds. (derives from: p1, p3)
2. [interval] Avoidance frequency falls within a 20-35% band after exposure therapy. (derives from: p2)
3. [directional] Higher perceived threat is associated with more avoidance. (derives from: p2)

Severity

• pred1: severity 1.0, risk 0.9
• pred2: severity 0.7, risk 0.7
• pred3: severity 0.3, risk 0.4

Passing a path writes the same markdown to disk with LF line endings, which is convenient when the document is committed to a repository or uploaded to a registry.

prereg_path <- tempfile(fileext = ".md")
invisible(tf_preregister(theory, path = prereg_path))
file.exists(prereg_path)
#> [1] TRUE

Appraising an amendment

Theories change. The question is whether a change is progressive, in that it predicts and then corroborates something new, or degenerating, in that it only adds assumptions to absorb anomalies. tf_appraise_amendment() compares an amended theory against its prior version and returns a verdict in those terms.

Here the prior is the panic-network fixture and the amended version is its 2026 revision, which adds one new prediction that is later corroborated and introduces no new ad-hoc assumptions.

v2 <- tf_read(
  system.file("fixtures/panic-network-2026-v2.theory.yaml", package = "theoryforge")
)

appraisal <- tf_appraise_amendment(v2, theory)
appraisal$verdict
#> [1] "progressive"
appraisal$new_predictions
#> [1] "pred4"
appraisal$corroborated_new
#> [1] "pred4"
appraisal$ad_hoc_assumptions
#> character(0)

A progressive verdict reflects new content that survived a test. Had the amendment instead added assumptions tied to specific anomalies without generating corroborated predictions, the verdict would record that as degenerating.

Compiling a measurement model

When constructs carry measurement indicators, tf_compile_sem() compiles them into lavaan model syntax. Constructs with indicators become latent measurement models with =~, and propositions become structural paths with ~, covariances with ~~ or annotated moderation lines.

cat(tf_compile_sem(theory))

# lavaan model generated by theoryforge for panic-network-2026
# Measurement model
c_arousal =~ heart_rate_variability + self_reported_arousal
c_perceived_threat =~ threat_appraisal_questionnaire
c_avoidance =~ behavioural_avoidance_task + diary_counts
# Structural model
c_perceived_threat ~ c_arousal
c_avoidance ~ c_perceived_threat
c_arousal ~ c_perceived_threat

The function returns a plain string and does not require lavaan to be installed. The output can be passed directly to lavaan::sem() once the data are available.

The reviewer audit bundle

tf_dossier() assembles a single markdown document for a reviewer or editor. It collects the header, the rigour-checklist table, the severity list, the provenance trail and the preregistration into one bundle, so a reader can follow the theory from its claims through to its planned tests in one place. The opening of the bundle, through the rigour-checklist table, renders below.

dossier <- tf_dossier(theory)
cat(strsplit(dossier, "\n## Severity", fixed = TRUE)[[1]][1])

theoryforge dossier: Network theory of panic disorder

• Theory ID: panic-network-2026
• Maturity: developing
• Aggregate rigour score: 84.8/100
• Gate: pass
• Blockers failed: 0

Rigour checklist

item	status	score	weight
falsifiability	pass	1.0	0.15
precision	pass	0.667	0.1
risk_severity	pass	0.567	0.1
parsimony	pass	0.667	0.08
non_redundancy	pass	0.909	0.1
construct_clarity	pass	1.0	0.08
scope	pass	1.0	0.06
logical_why	pass	1.0	0.08
causal_testability	pass	1.0	0.06
diagnosticity	pass	0.333	0.06
formalisation	pass	1.0	0.05
derivation_chain	pass	1.0	0.08

The severity list, the provenance trail and the preregistration follow in the full document, which is deterministic, so the same theory always produces the same dossier.

Rendering and depositing

tf_render_report() writes the dossier as a Quarto source document, ready to render to HTML or PDF. With render = FALSE it writes the .qmd without calling Quarto, so the step runs anywhere.

report_path <- tf_render_report(theory, tempfile(fileext = ".qmd"))
basename(report_path)
#> [1] "file1e9a3b09fc6e.qmd"

tf_osf_push() deposits the rendered report to an Open Science Framework node. It defaults to dry_run = TRUE, which assembles the request and returns it without sending anything, so the planned deposit can be inspected offline and without a token.

str(tf_osf_push(theory, node = "ab12c"))
#> List of 3
#>  $ dry_run: logi TRUE
#>  $ request:List of 4
#>   ..$ method       : chr "PUT"
#>   ..$ url          : chr "https://files.osf.io/v1/resources/ab12c/providers/osfstorage/?kind=file&name=panic-network-2026.dossier.md"
#>   ..$ filename     : chr "panic-network-2026.dossier.md"
#>   ..$ content_bytes: int 1644
#>  $ note   : chr "set dry_run=FALSE with a valid token and node to perform the upload"

The returned request records the method, the destination URL, the filename the deposit would create and the size of the dossier in bytes, so the step can be checked before anything leaves the machine. A live deposit needs dry_run = FALSE together with a token and the node id, and that is the only path that touches the network.

tf_osf_push(theory, node = "ab12c", token = Sys.getenv("OSF_PAT"), dry_run = FALSE)

Visualising the theory

tf_diagram() exports several views of the same object. The workflow view traces the lifecycle from constructs, through propositions and predictions, to the recorded test outcomes, so the path from a claim to its planned test is visible in one figure.

cat(tf_diagram(theory, "workflow"))

digraph workflow {
  graph [rankdir=LR, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
  node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
  edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
  subgraph cluster_build {
    label="building";
    style="rounded";
    color="#C4D1D9";
    fontcolor="#5B7285";
    "c_arousal" [label="Physiological\narousal", fillcolor="#E4F1F1", color="#1E7B7B"];
    "c_perceived_threat" [label="Perceived threat", fillcolor="#E4F1F1", color="#1E7B7B"];
    "c_avoidance" [label="Avoidance\nbehaviour", fillcolor="#E4F1F1", color="#1E7B7B"];
  }
  subgraph cluster_relate {
    label="propositions";
    style="rounded";
    color="#C4D1D9";
    fontcolor="#5B7285";
    "prop_p1" [label="p1\nincreases", fillcolor="#FBF1DC", color="#9C6B14"];
    "prop_p2" [label="p2\nincreases", fillcolor="#FBF1DC", color="#9C6B14"];
    "prop_p3" [label="p3\ncauses", fillcolor="#FBF1DC", color="#9C6B14"];
  }
  subgraph cluster_predict {
    label="predictions";
    style="rounded";
    color="#C4D1D9";
    fontcolor="#5B7285";
    "pred_pred1" [label="pred1\npoint", fillcolor="#E7EDF5", color="#33567A"];
    "pred_pred2" [label="pred2\ninterval", fillcolor="#E7EDF5", color="#33567A"];
    "pred_pred3" [label="pred3\ndirectional", fillcolor="#E7EDF5", color="#33567A"];
  }
  subgraph cluster_test {
    label="testing";
    style="rounded";
    color="#C4D1D9";
    fontcolor="#5B7285";
    "outcome_pred1" [label="pred1\npassed", fillcolor="#E5F2E7", color="#3E7A46"];
  }
  "c_arousal" -> "prop_p1";
  "c_perceived_threat" -> "prop_p2";
  "c_perceived_threat" -> "prop_p3";
  "prop_p1" -> "pred_pred1";
  "prop_p3" -> "pred_pred1";
  "prop_p2" -> "pred_pred2";
  "prop_p2" -> "pred_pred3";
  "pred_pred1" -> "outcome_pred1";
}

Rendered with tf_render_diagram(), which passes the DOT to the DiagrammeR engine, the same view reads as a figure.

cat('<div class="tf-figure tf-diagram">', tf_render_diagram(theory, "workflow", as = "svg"), '</div>', sep = "")

The context view places the theory among the scope conditions under which it is claimed to hold and the registered rivals it is meant to outpredict.

cat(tf_diagram(theory, "context"))

digraph context {
  graph [rankdir=LR, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
  node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
  edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
  "theory" [shape=ellipse, label="Network theory of\npanic disorder", fillcolor="#12283A", color="#12283A", fontcolor="#FFFFFF"];
  "c_arousal" [label="Physiological\narousal", fillcolor="#E4F1F1", color="#1E7B7B"];
  "theory" -> "c_arousal";
  "c_perceived_threat" [label="Perceived threat", fillcolor="#E4F1F1", color="#1E7B7B"];
  "theory" -> "c_perceived_threat";
  "c_avoidance" [label="Avoidance\nbehaviour", fillcolor="#E4F1F1", color="#1E7B7B"];
  "theory" -> "c_avoidance";
  "scope1" [shape=note, style="filled", label="adults", fillcolor="#FBF7EA", color="#B49B55"];
  "scope1" -> "theory" [style=dotted, label="holds within"];
  "scope2" [shape=note, style="filled", label="non-clinical\nbaseline", fillcolor="#FBF7EA", color="#B49B55"];
  "scope2" -> "theory" [style=dotted, label="holds within"];
  "scope3" [shape=note, style="filled", label="no beta-blocker\nmedication", fillcolor="#FBF7EA", color="#B49B55"];
  "scope3" -> "theory" [style=dotted, label="holds within"];
  "scope4" [shape=note, style="filled", label="waking hours", fillcolor="#FBF7EA", color="#B49B55"];
  "scope4" -> "theory" [style=dotted, label="holds within"];
  "scope5" [shape=note, style="filled", label="community sample", fillcolor="#FBF7EA", color="#B49B55"];
  "scope5" -> "theory" [style=dotted, label="holds within"];
  "alt_cognitive" [style="rounded,filled,dashed", label="Cognitive model of\npanic", fillcolor="#F1F1F1", color="#8A8A8A"];
  "theory" -> "alt_cognitive" [style=dashed, label="contrasts with"];
  "alt_biological" [style="rounded,filled,dashed", label="Biological model\nof panic", fillcolor="#F1F1F1", color="#8A8A8A"];
  "theory" -> "alt_biological" [style=dashed, label="contrasts with"];
}

cat('<div class="tf-figure tf-diagram">', tf_render_diagram(theory, "context", as = "svg"), '</div>', sep = "")

The pipeline view links each prediction to its recorded test outcome, so predictions still awaiting a test are visible as loose ends.

cat(tf_diagram(theory, "pipeline"))

digraph pipeline {
  graph [rankdir=LR, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
  node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
  edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
  "pred1" [label="pred1\npoint", fillcolor="#E7EDF5", color="#33567A"];
  "pred2" [label="pred2\ninterval", fillcolor="#E7EDF5", color="#33567A"];
  "pred3" [label="pred3\ndirectional", fillcolor="#E7EDF5", color="#33567A"];
  "result_pred1" [label="passed", fillcolor="#E5F2E7", color="#3E7A46"];
  "pred1" -> "result_pred1";
}

cat('<div class="tf-figure tf-diagram">', tf_render_diagram(theory, "pipeline", as = "svg"), '</div>', sep = "")

The provenance view draws the build log as a digraph, so the record of how the theory reached its current state travels with the object itself.

cat(tf_diagram(theory, "provenance"))

digraph provenance {
  graph [rankdir=TB, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
  node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
  edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
  "n1" [label="tf_construct\nRegistered three\nconstructs."];
  "n2" [label="tf_proposition\nLinked constructs into a\nfeedback network."];
  "n3" [label="tf_predict\nDerived three predictions\nfrom the propositions."];
  "n1" -> "n2";
  "n2" -> "n3";
}

cat('<div class="tf-figure tf-diagram">', tf_render_diagram(theory, "provenance", as = "svg"), '</div>', sep = "")

The venn view takes each construct’s boundary conditions as a set and returns an SVG showing where those scopes coincide and where they part company, with each region labelled by the number of conditions in it. It renders inline because the output is a self-contained image.

cat('<div class="tf-figure">', tf_diagram(theory, "venn"), '</div>', sep = "")

Two further views summarise the scoring. The rigour view draws the checklist as a status grid, colouring each item by its result and reporting the aggregate score and the gate.

cat('<div class="tf-figure">', tf_diagram(theory, "rigour"), '</div>', sep = "")

The severity view draws one bar per prediction, scaled by its computed severity, so the riskier tests stand out at a glance.

cat('<div class="tf-figure">', tf_diagram(theory, "severity"), '</div>', sep = "")

The development_roadmap view turns the same checklist into a worklist by keeping only the items that still fail or warn, and it orders that worklist the way the work should be done. Whatever gates the theory comes first, heaviest check first, and the advisory items follow. Each step carries its number, the criterion it is measured against and whether missing it blocks the gate or is merely advisory, so the figure says what to do next rather than only what went wrong. The hub names the theory and its current standing. The panic network passes every check, so its roadmap reduces to that hub and an all checks pass node, and the deliberately weak theory shipped alongside it shows the worklist in full.

weak <- tf_read(
  system.file("fixtures/weak-theory.theory.yaml", package = "theoryforge")
)
cat(tf_diagram(theory, "development_roadmap"))

digraph development_roadmap {
  graph [rankdir=TB, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
  node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
  edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
  "roadmap" [shape=ellipse, label="Network theory of\npanic disorder\nscore 84.8, gate pass", fillcolor="#12283A", color="#12283A", fontcolor="#FFFFFF"];
  "all_checks_pass" [label="all checks pass", fillcolor="#E5F2E7", color="#3E7A46"];
  "roadmap" -> "all_checks_pass";
}

cat(tf_diagram(weak, "development_roadmap"))

digraph development_roadmap {
  graph [rankdir=TB, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
  node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
  edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
  "roadmap" [shape=ellipse, label="An underspecified\nmotivation theory\nscore 12.0, gate blocked", fillcolor="#12283A", color="#12283A", fontcolor="#FFFFFF"];
  "falsifiability" [label="1. falsifiability\nAt least one\nprediction forbids an\nobservation\nblocks the gate", fillcolor="#F9E5E4", color="#B2453C"];
  "derivation_chain" [label="2. derivation_chain\nEach prediction is\ngraph-reachable from\npropositions\n(reachability only)\nblocks the gate", fillcolor="#F9E5E4", color="#B2453C"];
  "precision" [label="3. precision\nPredictions are\npoint/interval, not\nmerely directional\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
  "risk_severity" [label="4. risk_severity\nMean prediction\nseverity above\nthreshold\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
  "construct_clarity" [label="5. construct_clarity\nEvery construct has\ndefinition +\nmeasurement + boundary\nconditions\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
  "logical_why" [label="6. logical_why\nEach proposition\nstates a mechanism,\nnot just a correlation\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
  "scope" [label="7. scope\nBoundary conditions\nexplicitly stated\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
  "causal_testability" [label="8. causal_testability\nCausal relations\nexport to a DAG with\nderivable implications\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
  "diagnosticity" [label="9. diagnosticity\nAt least one\nprediction\ndiscriminates from a\nregistered alternative\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
  "formalisation" [label="10. formalisation\nA formal-model stub\nexists (warn-only at\nbuilding stage)\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
  "roadmap" -> "falsifiability";
  "falsifiability" -> "derivation_chain";
  "derivation_chain" -> "precision";
  { rank=same; "precision" -> "risk_severity" -> "construct_clarity" [style=invis]; }
  "precision" -> "logical_why" [style=invis];
  { rank=same; "logical_why" -> "scope" -> "causal_testability" [style=invis]; }
  "logical_why" -> "diagnosticity" [style=invis];
  { rank=same; "diagnosticity" -> "formalisation" [style=invis]; }
}

cat('<div class="tf-figure tf-diagram">', tf_render_diagram(weak, "development_roadmap", as = "svg"), '</div>', sep = "")

All of these views are deterministic. The DOT strings render with any Graphviz engine, and the SVG views open in any browser. Together with the nomological_net and causal_dag views introduced in the Get started vignette, they complete the set of diagram types that tf_diagram() exports, each documented in ?tf_diagram.