theoryforge: A theory you can check

Treat the theory as a checked document

Structural validation checks required fields and allowed values. Full validation also follows identifiers across the document, so a prediction cannot claim to derive from a proposition that does not exist.

tf_validate(switching, full = TRUE)

check <- tf_check(switching)
c(score = check$aggregate_score,
  gate = check$gate,
  blockers_failed = check$n_blockers_failed)

#>           score            gate blockers_failed 
#>          "85.1"          "pass"             "0"

The checklist is based on features that make a theory appraisable, including explicit prohibitions and predictions linked to propositions (Meehl, 1990). Its score describes completeness of specification, not truth. A well-specified theory can be wrong, while a promising informal account can score poorly because it is not yet explicit enough to test.

Read the graph

The theory’s propositions form a causal graph. Because the source is structured data, the same graph can be rendered for a reader or exported for another program.

flowchart LR
  c_sensorimotor_experience["Sensorimotor experience with a concept"]
  c_modality_activation["Modality-specific perceptual activation"]
  c_switch_cost["Cost of switching modality between consecutive trials"]
  c_conceptual_access["Ease of conceptual access"]
  c_lexical_familiarity["Lexical familiarity with the word form"]
  c_sensorimotor_experience -->|increases| c_modality_activation
  c_modality_activation -->|increases| c_switch_cost
  c_modality_activation -->|increases| c_conceptual_access
  c_lexical_familiarity -->|increases| c_conceptual_access

An acyclic causal graph does more than display arrows. It implies conditional independencies: associations that should disappear, or never arise, if the graph is correct. Those claims define what the theory forbids in observed data.

The graph contains four arrows and entails six testable independencies. The most discriminating one is that sensorimotor experience and lexical familiarity should be uncorrelated, because the only path between them meets at a collider. An observed correlation between the two, with nothing conditioned on, would count against the graph.

Give the claims a discriminating test

The example below generates two data sets. The first follows the theory’s graph. The second represents a rival account in which sensorimotor experience also contributes to lexical familiarity, with all other paths held constant.

All six implied independencies are compatible with data generated from the theory. The table isolates the one claim that fails in the rival world: sensorimotor experience and lexical familiarity become strongly associated, while the other claims remain close to zero. The pattern identifies which part of the graph conflicts with the data. A single global score would not provide that diagnosis.

Preserve the result with the theory

The dossier brings the definitions, propositions, predictions, checks and provenance into one Markdown record. Its checksum identifies the exact state of the theory independently of whether R or Python produced it. For readability, the compact summary below shows only the checksum prefix.

Limits

Formalisation moves judgement into the open without removing it. Researchers still decide which constructs and arrows belong in the graph, how they are measured and whether an observed departure is scientifically important. The checklist is a structural and lexical screen, and its score must not be read as evidence that the theory is true. Feedback loops also require methods beyond the conditional-independence analysis used here.

The package documentation covers structural equation model compilation, severity scoring, preregistration, amendment appraisal, version differences and OSF deposits. Those tools are useful after the core move shown here: write the theory in a form that exposes what it claims and what observations would count against it.