theoryforge: A theory you can check

Tue, 01 Sep 2026 00:00:00 +0000

A verbal theory can sound precise while leaving its structure uncertain. Its constructs may overlap, a prediction may not follow from any proposition and an amendment may protect the theory without adding a new risk. These are properties of the theory’s specification, yet ordinary prose gives software nothing to inspect.

I wrote theoryforge to store that specification as a small YAML or JSON document. The document records constructs, propositions, predictions, alternatives, versions and provenance under a published schema. The package can then validate references between those parts, derive implications from the causal graph and produce a dossier for review. Matching R and Python implementations use the same format.

The example is a theory of modality switching in grounded conceptual processing. People are slower to verify a conceptual property after the preceding trial involved a different perceptual modality (Pecher et al., 2003). This effect has been interpreted as evidence that conceptual access recruits modality-specific systems (Barsalou, 1999).

library(theoryforge)
packageVersion('theoryforge')

#> [1] '0.6.0'

switching <- tf_read(tf_example_path('modality-switching.theory.yaml'))
c(id = switching$id,
maturity = switching$maturity,
constructs = length(switching$constructs),
propositions = length(switching$propositions),
predictions = length(switching$predictions))

#> id maturity constructs
#> "modality-switching-2026" "developing" "5"
#> propositions predictions
#> "4" "4"

Treat the theory as a checked document

Structural validation checks required fields and allowed values. Full validation also follows identifiers across the document, so a prediction cannot claim to derive from a proposition that does not exist.

tf_validate(switching, full = TRUE)
check <- tf_check(switching)
c(score = check$aggregate_score,
gate = check$gate,
blockers_failed = check$n_blockers_failed)

#> score gate blockers_failed
#> "85.1" "pass" "0"

The checklist is based on features that make a theory appraisable, including explicit prohibitions and predictions linked to propositions (Meehl, 1990). Its score describes completeness of specification, not truth. A well-specified theory can be wrong, while a promising informal account can score poorly because it is not yet explicit enough to test.

Read the graph

The theory’s propositions form a causal graph. Because the source is structured data, the same graph can be rendered for a reader or exported for another program.

flowchart LR
c_sensorimotor_experience["Sensorimotor experience with a concept"]
c_modality_activation["Modality-specific perceptual activation"]
c_switch_cost["Cost of switching modality between consecutive trials"]
c_conceptual_access["Ease of conceptual access"]
c_lexical_familiarity["Lexical familiarity with the word form"]
c_sensorimotor_experience -->|increases| c_modality_activation
c_modality_activation -->|increases| c_switch_cost
c_modality_activation -->|increases| c_conceptual_access
c_lexical_familiarity -->|increases| c_conceptual_access

An acyclic causal graph does more than display arrows. It implies conditional independencies: associations that should disappear, or never arise, if the graph is correct. Those claims define what the theory forbids in observed data.

The graph contains four arrows and entails six testable independencies. The most discriminating one is that sensorimotor experience and lexical familiarity should be uncorrelated, because the only path between them meets at a collider. An observed correlation between the two, with nothing conditioned on, would count against the graph.

Give the claims a discriminating test

The example below generates two data sets. The first follows the theory’s graph. The second represents a rival account in which sensorimotor experience also contributes to lexical familiarity, with all other paths held constant.

World	Correlation	p value
Data generated from the theory	-0.009	0.56
Data generated from the rival	0.445	<1e-99

All six implied independencies are compatible with data generated from the theory. The table isolates the one claim that fails in the rival world: sensorimotor experience and lexical familiarity become strongly associated, while the other claims remain close to zero. The pattern identifies which part of the graph conflicts with the data. A single global score would not provide that diagnosis.

Preserve the result with the theory

The dossier brings the definitions, propositions, predictions, checks and provenance into one Markdown record. Its checksum identifies the exact state of the theory independently of whether R or Python produced it. For readability, the compact summary below shows only the checksum prefix.

Detail	Value
Rigour score	85.1/100
Gate	pass
SHA-256 prefix	4c07a9235e37

Limits

Formalisation moves judgement into the open without removing it. Researchers still decide which constructs and arrows belong in the graph, how they are measured and whether an observed departure is scientifically important. The checklist is a structural and lexical screen, and its score must not be read as evidence that the theory is true. Feedback loops also require methods beyond the conditional-independence analysis used here.

The package documentation covers structural equation model compilation, severity scoring, preregistration, amendment appraisal, version differences and OSF deposits. Those tools are useful after the core move shown here: write the theory in a form that exposes what it claims and what observations would count against it.

References

Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22(4), 577–660. https://doi.org/10.1017/S0140525X99002149

Meehl, P. E. (1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it. Psychological Inquiry, 1(2), 108–141. https://doi.org/10.1207/s15327965pli0102_1

Pecher, D., Zeelenberg, R., & Barsalou, L. W. (2003). Verifying different-modality properties for concepts produces switching costs. Psychological Science, 14(2), 119–124. https://doi.org/10.1111/1467-9280.t01-1-01429