Workflow modes
theoryforge organises work into three modes: BUILDING, DEVELOPMENT and
TESTING. A theory is a single versioned object that moves through these
modes as it matures. Each mode is a set of methods on the Theory object,
so the same artefact carries from a first draft to a preregistered test.
This page assumes you have already read a theory as in Getting started; see Methodological foundations for the rationale and the exact computation behind the rigour checklist, severity and amendment-appraisal rules demonstrated below.
Throughout, the package is imported as tf, and fixtures names the
repository's fixture directory, as on the Getting started
page. Every result shown below is produced by running the code when this page is
built.
import theoryforge as tf
BUILDING
BUILDING assembles a theory from scratch. tf.new_theory() returns an empty
Theory, and the add_* methods append constructs, propositions, and
predictions. Each method returns the same object, so calls chain. Every
addition is recorded in a provenance log, which gives a step-by-step account
of how the theory was assembled.
t = (
tf.new_theory("panic_demo", "A demonstration theory of panic")
.add_construct(
"arousal",
"Physiological arousal",
"bodily signs of sympathetic activation",
measurement=["heart_rate", "skin_conductance"],
boundary_conditions=["adults"],
)
.add_construct(
"catastrophic_interpretation",
"Catastrophic interpretation",
"appraisal of bodily sensations as dangerous",
measurement=["bsiq"],
)
.add_proposition(
"p1",
"arousal",
"catastrophic_interpretation",
"increases",
mechanism="rising arousal is read as evidence of threat",
)
.add_prediction(
"pred1",
"higher arousal predicts more catastrophic interpretation",
"directional",
derives_from=["p1"],
)
)
The positional arguments follow the schema. add_construct(id, label,
definition) takes optional measurement and boundary_conditions lists.
add_proposition(id, frm, to, relation) takes an optional mechanism,
where relation is one of increases, decreases, moderates,
mediates, causes or associates (frm stands in for the schema's
from field, since from is a reserved word in Python). add_prediction(id, statement,
type) takes optional derives_from and diagnostic_vs lists, where type
is one of point, interval, directional or existence.
The provenance log is held under t.data["provenance"]. Each entry records
the action and the identifier it affected.
for step in t.data["provenance"]:
print(step["step"], step["action"], step["detail"])
1 tf_theory panic_demo
2 tf_add_construct arousal
3 tf_add_construct catastrophic_interpretation
4 tf_add_proposition p1
5 tf_add_prediction pred1
Once the object is assembled, t.validate() confirms it satisfies the
schema's required fields, returning True silently, and t.report() returns
the rigour checklist. The report opens with the aggregate score and the gate,
then gives one entry per checklist item.
t.validate()
print(t.report("json"))
{
"theory_id": "panic_demo",
"schema_version": "1.0",
"maturity": "building",
"aggregate_score": 57.6,
"gate": "pass",
"n_blockers_failed": 0,
"items": [
{
"id": "falsifiability",
"status": "pass",
"score": 1.0,
"weight": 0.15,
"severity_if_fail": "blocker",
"citation": "Popper (1959); Bacharach (1989)"
},
{
"id": "precision",
"status": "warn",
"score": 0.0,
"weight": 0.1,
"severity_if_fail": "warning",
"citation": "Meehl (1967, 1990)"
},
{
"id": "risk_severity",
"status": "warn",
"score": 0.0,
"weight": 0.1,
"severity_if_fail": "warning",
"citation": "Mayo (2018); Meehl (1990)"
},
{
"id": "parsimony",
"status": "pass",
"score": 1.0,
"weight": 0.08,
"severity_if_fail": "warning",
"citation": "Forster & Sober (1994); Lakatos (1970)"
},
{
"id": "non_redundancy",
"status": "pass",
"score": 0.857,
"weight": 0.1,
"severity_if_fail": "warning",
"citation": "Kelley (1927); Le et al. (2010); Lawson & Robins (2021)"
},
{
"id": "construct_clarity",
"status": "warn",
"score": 0.5,
"weight": 0.08,
"severity_if_fail": "warning",
"citation": "Suddaby (2010); Cronbach & Meehl (1955); Flake & Fried (2020)"
},
{
"id": "scope",
"status": "warn",
"score": 0.0,
"weight": 0.06,
"severity_if_fail": "warning",
"citation": "Whetten (1989); Bacharach (1989)"
},
{
"id": "logical_why",
"status": "pass",
"score": 1.0,
"weight": 0.08,
"severity_if_fail": "warning",
"citation": "Sutton & Staw (1995); Whetten (1989)"
},
{
"id": "causal_testability",
"status": "pass",
"score": 1.0,
"weight": 0.06,
"severity_if_fail": "warning",
"citation": "Textor et al. (2016); Eronen & Bringmann (2021)"
},
{
"id": "diagnosticity",
"status": "warn",
"score": 0.0,
"weight": 0.06,
"severity_if_fail": "warning",
"citation": "Platt (1964); Fiedler (2017)"
},
{
"id": "formalisation",
"status": "warn",
"score": 0.0,
"weight": 0.05,
"severity_if_fail": "warning",
"citation": "Robinaugh et al. (2021); Guest & Martin (2021)"
},
{
"id": "derivation_chain",
"status": "pass",
"score": 1.0,
"weight": 0.08,
"severity_if_fail": "blocker",
"citation": "Scheel et al. (2021); Szollosi et al. (2020)"
}
]
}
DEVELOPMENT
DEVELOPMENT compares two versions of a theory and judges whether an amendment is an improvement. The appraisal operationalises the Lakatosian distinction between progressive and degenerating problem shifts. An amendment is progressive when it yields newly corroborated predictions without resorting to ad-hoc immunising assumptions.
Call appraise_amendment on the newer version, passing the prior version as
the argument.
v1 = tf.read(fixtures / "panic-network.theory.yaml")
v2 = tf.read(fixtures / "panic-network-2026-v2.theory.yaml")
result = v2.appraise_amendment(v1)
print(result["verdict"])
progressive
The return value is a dictionary with four keys.
{
"verdict": "progressive", # 'progressive', 'degenerating', or 'neutral'
"new_predictions": ["pred4"], # prediction ids present in v2 but not v1
"corroborated_new": ["pred4"], # of those, the ones with a passed test outcome
"ad_hoc_assumptions": [], # new assumptions that protect untested predictions
}
The verdict depends on the recorded test outcomes and any new auxiliary assumptions in the newer version. The rule is as follows.
progressive: at least one new prediction is corroborated and no ad-hoc assumptions were added.degenerating: at least one ad-hoc assumption was added and no new prediction is corroborated.neutral: any other combination.
A new prediction counts as corroborated when the newer version carries a
matching entry under test_outcomes with passed set to True. A new
auxiliary assumption counts as ad-hoc when it has an added_for reason but
none of the predictions it claims to protect has a passing test outcome.
TESTING
TESTING prepares a theory for empirical evaluation. It scores how risky each prediction is, exports a preregistration document, and assembles a reviewer-facing audit bundle.
Severity
severity() returns one entry per prediction, in file order, giving the
base risk implied by the prediction type and the computed severity after the
rubric's adjustments.
for s in t.severity():
print(s["prediction_id"], s["risk_score"], s["computed_severity"])
pred1 0.4 0.3
Each entry has the shape below.
{
"prediction_id": "pred1",
"type": "directional",
"risk_score": 0.4, # base risk from the prediction type
"computed_severity": 0.3, # after the directional discount and any diagnostic bonus
}
Base risk rises with the strength of the claim: existence 0.1,
directional 0.4, interval 0.7, point 0.9. Directional predictions
carry a discount for the ambient correlations expected in the field. A
prediction that discriminates the theory from a named alternative, through
its diagnostic_vs field, earns a small severity bonus.
Preregistration
preregister() renders a Markdown preregistration document. It lists the
hypotheses with their derivations and a severity line for each prediction.
Called without an argument it returns the text.
text = t.preregister()
print(text)
# Preregistration: A demonstration theory of panic
- Theory ID: panic_demo
- Schema version: 1.0
- Maturity: building
- Derivation chain verified: yes
## Hypotheses
1. [directional] higher arousal predicts more catastrophic interpretation (derives from: p1)
## Severity
- pred1: severity 0.3, risk 0.4
Passing a path writes the document to disk as well as returning it.
t.preregister("panic-prereg.md")
Audit bundle
dossier() assembles a single Markdown document for reviewers. It composes
the rigour checklist, the severity table, the provenance log and the
preregistration into one artefact. The output is deterministic, so it can be
committed or attached to a submission. The bundle opens with the header and the
rigour checklist, and the severity table, provenance log and preregistration
follow.
print(t.dossier())
# theoryforge dossier: A demonstration theory of panic
- Theory ID: panic_demo
- Maturity: building
- Aggregate rigour score: 57.6/100
- Gate: pass
- Blockers failed: 0
## Rigour checklist
| item | status | score | weight |
| --- | --- | --- | --- |
| falsifiability | pass | 1.0 | 0.15 |
| precision | warn | 0.0 | 0.1 |
| risk_severity | warn | 0.0 | 0.1 |
| parsimony | pass | 1.0 | 0.08 |
| non_redundancy | pass | 0.857 | 0.1 |
| construct_clarity | warn | 0.5 | 0.08 |
| scope | warn | 0.0 | 0.06 |
| logical_why | pass | 1.0 | 0.08 |
| causal_testability | pass | 1.0 | 0.06 |
| diagnosticity | warn | 0.0 | 0.06 |
| formalisation | warn | 0.0 | 0.05 |
| derivation_chain | pass | 1.0 | 0.08 |
## Severity
- pred1: severity 0.3, risk 0.4
## Provenance
1. tf_theory: panic_demo
2. tf_add_construct: arousal
3. tf_add_construct: catastrophic_interpretation
4. tf_add_proposition: p1
5. tf_add_prediction: pred1
## Preregistration
# Preregistration: A demonstration theory of panic
- Theory ID: panic_demo
- Schema version: 1.0
- Maturity: building
- Derivation chain verified: yes
## Hypotheses
1. [directional] higher arousal predicts more catastrophic interpretation (derives from: p1)
## Severity
- pred1: severity 0.3, risk 0.4
compile_sem() translates the structural content into lavaan model syntax.
Constructs with measurement indicators become a latent measurement model
with =~, and directed propositions become structural paths with ~.
print(t.compile_sem())
# lavaan model generated by theoryforge for panic_demo
# Measurement model
arousal =~ heart_rate + skin_conductance
catastrophic_interpretation =~ bsiq
# Structural model
catastrophic_interpretation ~ arousal
These three modes share one object, so a theory built with the BUILDING methods can be appraised under DEVELOPMENT and then carried into TESTING without conversion.
Visualising the theory
diagram() exports several views of the same object. The graph views return
Graphviz DOT or dagitty text; three further views are returned directly as SVG
and render inline. These examples use the repository's panic-network fixture
(see Getting started for where the fixture files live),
which carries the test outcomes and scope conditions the richer views draw on.
t = tf.read(fixtures / "panic-network.theory.yaml")
The workflow view traces the lifecycle from constructs, through propositions and
predictions, to the recorded test outcomes.
print(t.diagram("workflow"))
digraph workflow {
graph [rankdir=LR, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
subgraph cluster_build {
label="building";
style="rounded";
color="#C4D1D9";
fontcolor="#5B7285";
"c_arousal" [label="Physiological\narousal", fillcolor="#E4F1F1", color="#1E7B7B"];
"c_perceived_threat" [label="Perceived threat", fillcolor="#E4F1F1", color="#1E7B7B"];
"c_avoidance" [label="Avoidance\nbehaviour", fillcolor="#E4F1F1", color="#1E7B7B"];
}
subgraph cluster_relate {
label="propositions";
style="rounded";
color="#C4D1D9";
fontcolor="#5B7285";
"prop_p1" [label="p1\nincreases", fillcolor="#FBF1DC", color="#9C6B14"];
"prop_p2" [label="p2\nincreases", fillcolor="#FBF1DC", color="#9C6B14"];
"prop_p3" [label="p3\ncauses", fillcolor="#FBF1DC", color="#9C6B14"];
}
subgraph cluster_predict {
label="predictions";
style="rounded";
color="#C4D1D9";
fontcolor="#5B7285";
"pred_pred1" [label="pred1\npoint", fillcolor="#E7EDF5", color="#33567A"];
"pred_pred2" [label="pred2\ninterval", fillcolor="#E7EDF5", color="#33567A"];
"pred_pred3" [label="pred3\ndirectional", fillcolor="#E7EDF5", color="#33567A"];
}
subgraph cluster_test {
label="testing";
style="rounded";
color="#C4D1D9";
fontcolor="#5B7285";
"outcome_pred1" [label="pred1\npassed", fillcolor="#E5F2E7", color="#3E7A46"];
}
"c_arousal" -> "prop_p1";
"c_perceived_threat" -> "prop_p2";
"c_perceived_threat" -> "prop_p3";
"prop_p1" -> "pred_pred1";
"prop_p3" -> "pred_pred1";
"prop_p2" -> "pred_pred2";
"prop_p2" -> "pred_pred3";
"pred_pred1" -> "outcome_pred1";
}
With the optional render extra (pip install theoryforge[render]), the same
view renders without leaving Python: render_diagram(t, "workflow") wraps the
DOT in a graphviz.Source, which displays inline in a notebook and writes
image files through its render method. Rendered, the workflow view reads as
a figure.
The context view places the theory among the scope conditions under which it is
claimed to hold and the registered rivals it is meant to outpredict.
print(t.diagram("context"))
digraph context {
graph [rankdir=LR, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
"theory" [shape=ellipse, label="Network theory of\npanic disorder", fillcolor="#12283A", color="#12283A", fontcolor="#FFFFFF"];
"c_arousal" [label="Physiological\narousal", fillcolor="#E4F1F1", color="#1E7B7B"];
"theory" -> "c_arousal";
"c_perceived_threat" [label="Perceived threat", fillcolor="#E4F1F1", color="#1E7B7B"];
"theory" -> "c_perceived_threat";
"c_avoidance" [label="Avoidance\nbehaviour", fillcolor="#E4F1F1", color="#1E7B7B"];
"theory" -> "c_avoidance";
"scope1" [shape=note, style="filled", label="adults", fillcolor="#FBF7EA", color="#B49B55"];
"scope1" -> "theory" [style=dotted, label="holds within"];
"scope2" [shape=note, style="filled", label="non-clinical\nbaseline", fillcolor="#FBF7EA", color="#B49B55"];
"scope2" -> "theory" [style=dotted, label="holds within"];
"scope3" [shape=note, style="filled", label="no beta-blocker\nmedication", fillcolor="#FBF7EA", color="#B49B55"];
"scope3" -> "theory" [style=dotted, label="holds within"];
"scope4" [shape=note, style="filled", label="waking hours", fillcolor="#FBF7EA", color="#B49B55"];
"scope4" -> "theory" [style=dotted, label="holds within"];
"scope5" [shape=note, style="filled", label="community sample", fillcolor="#FBF7EA", color="#B49B55"];
"scope5" -> "theory" [style=dotted, label="holds within"];
"alt_cognitive" [style="rounded,filled,dashed", label="Cognitive model of\npanic", fillcolor="#F1F1F1", color="#8A8A8A"];
"theory" -> "alt_cognitive" [style=dashed, label="contrasts with"];
"alt_biological" [style="rounded,filled,dashed", label="Biological model\nof panic", fillcolor="#F1F1F1", color="#8A8A8A"];
"theory" -> "alt_biological" [style=dashed, label="contrasts with"];
}
The pipeline view links each prediction to its recorded test outcome, so
predictions still awaiting a test are visible as loose ends.
print(t.diagram("pipeline"))
digraph pipeline {
graph [rankdir=LR, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
"pred1" [label="pred1\npoint", fillcolor="#E7EDF5", color="#33567A"];
"pred2" [label="pred2\ninterval", fillcolor="#E7EDF5", color="#33567A"];
"pred3" [label="pred3\ndirectional", fillcolor="#E7EDF5", color="#33567A"];
"result_pred1" [label="passed", fillcolor="#E5F2E7", color="#3E7A46"];
"pred1" -> "result_pred1";
}
The provenance view draws the build log as a digraph, so the record of how
the theory reached its current state travels with the object itself.
print(t.diagram("provenance"))
digraph provenance {
graph [rankdir=TB, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
"n1" [label="tf_construct\nRegistered three\nconstructs."];
"n2" [label="tf_proposition\nLinked constructs into a\nfeedback network."];
"n3" [label="tf_predict\nDerived three predictions\nfrom the propositions."];
"n1" -> "n2";
"n2" -> "n3";
}
The venn view takes each construct's boundary conditions as a set and draws the
first three of them as overlapping discs, so the figure shows where construct
scopes coincide and where they part company. Each region is labelled with the
number of conditions that fall in it.
print(t.diagram("venn")) # construct scope overlap
print(t.diagram("rigour")) # the rigour checklist as a status grid
print(t.diagram("severity")) # per-prediction severity bars
The rigour view draws the checklist as a status grid, colouring each item by its
result and reporting the aggregate score and the gate.
The severity view draws one bar per prediction, scaled by its computed severity,
so the riskier tests stand out at a glance.
The development_roadmap view turns the same checklist into a worklist by
keeping only the items that still fail or warn, and it orders that worklist the
way the work should be done. Whatever gates the theory comes first, heaviest
check first, and the advisory items follow. Each step carries its number, the
criterion it is measured against and whether missing it blocks the gate or is
merely advisory, so the figure says what to do next rather than only what went
wrong. The hub names the theory and its current standing. The panic network
passes every check, so its roadmap reduces to that hub and an all checks pass
node, and the deliberately weak fixture shipped alongside it shows the worklist
in full.
print(tf.read(fixtures / "weak-theory.theory.yaml").diagram("development_roadmap"))
digraph development_roadmap {
graph [rankdir=TB, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
"roadmap" [shape=ellipse, label="An underspecified\nmotivation theory\nscore 12.0, gate blocked", fillcolor="#12283A", color="#12283A", fontcolor="#FFFFFF"];
"falsifiability" [label="1. falsifiability\nAt least one\nprediction forbids an\nobservation\nblocks the gate", fillcolor="#F9E5E4", color="#B2453C"];
"derivation_chain" [label="2. derivation_chain\nEach prediction is\ngraph-reachable from\npropositions\n(reachability only)\nblocks the gate", fillcolor="#F9E5E4", color="#B2453C"];
"precision" [label="3. precision\nPredictions are\npoint/interval, not\nmerely directional\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
"risk_severity" [label="4. risk_severity\nMean prediction\nseverity above\nthreshold\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
"construct_clarity" [label="5. construct_clarity\nEvery construct has\ndefinition +\nmeasurement + boundary\nconditions\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
"logical_why" [label="6. logical_why\nEach proposition\nstates a mechanism,\nnot just a correlation\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
"scope" [label="7. scope\nBoundary conditions\nexplicitly stated\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
"causal_testability" [label="8. causal_testability\nCausal relations\nexport to a DAG with\nderivable implications\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
"diagnosticity" [label="9. diagnosticity\nAt least one\nprediction\ndiscriminates from a\nregistered alternative\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
"formalisation" [label="10. formalisation\nA formal-model stub\nexists (warn-only at\nbuilding stage)\nadvisory", fillcolor="#FBF1DC", color="#9C6B14"];
"roadmap" -> "falsifiability";
"falsifiability" -> "derivation_chain";
"derivation_chain" -> "precision";
{ rank=same; "precision" -> "risk_severity" -> "construct_clarity" [style=invis]; }
"precision" -> "logical_why" [style=invis];
{ rank=same; "logical_why" -> "scope" -> "causal_testability" [style=invis]; }
"logical_why" -> "diagnosticity" [style=invis];
{ rank=same; "diagnosticity" -> "formalisation" [style=invis]; }
}
Together with the nomological_net and causal_dag views shown in
Getting started, these complete the set of diagram types
that diagram() exports, each documented in the
API reference.
Simulation
simulate() treats each construct as a state variable and integrates the signed
proposition network as a linear dynamical system with fixed-step (Euler) updates.
The trajectory is fully deterministic.
The example below adds a regulating edge to the panic structure: arousal raises threat, threat raises avoidance, and avoidance in turn decreases arousal. The negative coupling breaks the symmetry between the states, so the qualitative dynamics the network implies are visible in the trajectory.
s = (
tf.new_theory("regulation_demo", "Arousal regulated by avoidance")
.add_construct("arousal", "Physiological arousal", "bodily activation")
.add_construct("threat", "Perceived threat", "appraised danger")
.add_construct("avoidance", "Avoidance behaviour", "protective withdrawal")
.add_proposition("p1", "arousal", "threat", "increases")
.add_proposition("p2", "threat", "avoidance", "increases")
.add_proposition("p3", "avoidance", "arousal", "decreases")
)
sim = s.simulate(steps=5)
print(sim["states"]) # construct ids, in file order
for row in sim["trajectory"]: # the initial state, then the five Euler steps
print(row)
['arousal', 'threat', 'avoidance']
[1.0, 1.0, 1.0]
[0.85, 1.05, 1.05]
[0.7025, 1.0825, 1.1025]
[0.557125, 1.098625, 1.155625]
[0.413706, 1.099406, 1.207706]
[0.27225, 1.085807, 1.257262]
Arousal falls from the first step, pushed down by the negative coupling from avoidance on top of the damping. Threat climbs to a peak at step 4 and turns down at step 5, once the falling arousal no longer sustains it. Avoidance is still rising at the end of the window, and it is the last of the three to turn because it keeps integrating a threat level that stays high across all five steps.