Building, checking and diagramming a theory • theoryforge

theoryforge treats a scientific theory as a versioned, machine-checkable object. This vignette walks through the core loop of building a theory, checking its rigour and diagramming it (see the package overview for the R/Python parity guarantee).

library(theoryforge)

Building a theory

A theory is built incrementally with the BUILDING-mode verbs. Each verb appends to the theory and records a provenance entry, and each returns the theory, so the steps compose with the native pipe.

theory <- tf_theory("panic-network", "A network theory of panic") |>
  tf_add_construct("c_arousal", "Physiological arousal",
                   "Bodily activation in response to a stressor.",
                   measurement = "heart rate variability",
                   boundary_conditions = "awake adults") |>
  tf_add_construct("c_threat", "Perceived threat",
                   "Appraised danger of bodily sensations.",
                   measurement = "self-report appraisal scale",
                   boundary_conditions = "awake adults") |>
  tf_add_proposition(
    "p1", "c_arousal", "c_threat", "causes",
    mechanism = "Activation raises the salience of threat cues."
  ) |>
  tf_add_prediction("h1", "Arousal raises threat appraisal by a fixed amount.",
                    "point", derives_from = "p1")

isTRUE(tf_validate(theory)) # structural checks: required fields and enums

[1] TRUE

# also ids, cross-references and the rest of the schema
isTRUE(tf_validate(theory, full = TRUE))

[1] TRUE

tf_validate() returns TRUE invisibly on success and stops with a message listing every problem otherwise, which is why the calls above are wrapped in isTRUE() to show the result. With full = TRUE it additionally checks referential integrity, that ids are unique and every cross-reference points to a declared id. It then checks the rest of the schema, such as the evidence directions, a logical passed on each test outcome and the type of every other field. It does not require any optional dependency.

The failure path is the more informative one. Pointing a prediction at a proposition that was never declared leaves the structural pass untouched, since every required field is still present and well formed, so only full = TRUE reports the problem.

broken <- theory
broken$predictions[[1]]$derives_from <- "p_missing"
tf_validate(broken, full = TRUE)

Error:
! invalid theory object: prediction[0] derives_from 'p_missing' is not a known proposition

The building vocabulary has two further verbs. tf_add_assumption() records an auxiliary assumption, and tf_set_formal_model() attaches a formal-model reference. Applied to a copy of the theory, they leave the original untouched.

extended <- theory |>
  tf_add_assumption("a1", "Arousal is measured at rest.", added_for = "h1") |>
  tf_set_formal_model("sem", spec_ref = "panic-sem.lavaan")

Every verb appends a provenance entry, recording the step, the action and its detail, so the record of how a theory was built travels with the object.

do.call(rbind, lapply(extended$provenance, as.data.frame))

  step              action        detail
1    1           tf_theory panic-network
2    2    tf_add_construct     c_arousal
3    3    tf_add_construct      c_threat
4    4  tf_add_proposition            p1
5    5   tf_add_prediction            h1
6    6   tf_add_assumption            a1
7    7 tf_set_formal_model           sem

A theory can be written to and read back from disk. The format follows the file extension (.json for JSON, otherwise YAML).

path <- tempfile(fileext = ".yaml")
tf_write(theory, path)
roundtrip <- tf_read(path)
identical(roundtrip$id, theory$id)

[1] TRUE

Checking rigour

tf_check() runs the 12-item rigour checklist defined in the vendored rigor_checklist.yaml.

report <- tf_check(theory)

report$aggregate_score   # weighted score, 0-100, rounded to 1 dp

[1] 87

report$gate              # "pass", "blocked", or "advisory" (draft maturity)

[1] "pass"

report$n_blockers_failed # count of failed blocker items

[1] 0

# Per-item detail (checklist order preserved):
report$items[[1]]$id      # "falsifiability"

[1] "falsifiability"

report$items[[1]]$status  # "pass" / "warn" / "fail"

[1] "pass"

report$items[[1]]$score   # numeric in [0, 1]

[1] 1

An item with nothing to assess, such as the redundancy screen of a theory with a single construct, has the status "n/a" and a NULL score. It is left out of the aggregate, the weighted mean of the applicable items, and report$coverage gives their share of the checklist’s weight.

Render it as JSON (valid, pretty-printed) or as an HTML fragment. The JSON string is an artefact in its own right, shown here verbatim:

cat(tf_report(theory, format = "json"))

{
  "theory_id": "panic-network",
  "schema_version": "1.0",
  "checklist_version": "2.0",
  "maturity": "building",
  "aggregate_score": 87,
  "coverage": 0.92,
  "gate": "pass",
  "n_blockers_failed": 0,
  "items": [
    {
      "id": "falsifiability",
      "status": "pass",
      "score": 1,
      "weight": 0.15,
      "severity_if_fail": "blocker",
      "citation": "Popper (1959); Bacharach (1989)"
    },
    {
      "id": "precision",
      "status": "pass",
      "score": 1,
      "weight": 0.1,
      "severity_if_fail": "warning",
      "citation": "Meehl (1967, 1990)"
    },
    {
      "id": "risk_severity",
      "status": "pass",
      "score": 0.9,
      "weight": 0.1,
      "severity_if_fail": "warning",
      "citation": "Popper (1959); Meehl (1967, 1990)"
    },
    {
      "id": "parsimony",
      "status": "n/a",
      "score": null,
      "weight": 0.08,
      "severity_if_fail": "warning",
      "citation": "Lakatos (1970); Meehl (1990)"
    },
    {
      "id": "non_redundancy",
      "status": "pass",
      "score": 1,
      "weight": 0.1,
      "severity_if_fail": "warning",
      "citation": "Kelley (1927); Le et al. (2010); Lawson & Robins (2021)"
    },
    {
      "id": "construct_clarity",
      "status": "pass",
      "score": 1,
      "weight": 0.08,
      "severity_if_fail": "warning",
      "citation": "Suddaby (2010); Cronbach & Meehl (1955); Flake & Fried (2020)"
    },
    {
      "id": "scope",
      "status": "pass",
      "score": 1,
      "weight": 0.06,
      "severity_if_fail": "warning",
      "citation": "Whetten (1989); Bacharach (1989)"
    },
    {
      "id": "logical_why",
      "status": "pass",
      "score": 1,
      "weight": 0.08,
      "severity_if_fail": "warning",
      "citation": "Sutton & Staw (1995); Whetten (1989)"
    },
    {
      "id": "causal_testability",
      "status": "pass",
      "score": 1,
      "weight": 0.06,
      "severity_if_fail": "warning",
      "citation": "Textor et al. (2016); Eronen & Bringmann (2021)"
    },
    {
      "id": "diagnosticity",
      "status": "warn",
      "score": 0,
      "weight": 0.06,
      "severity_if_fail": "warning",
      "citation": "Platt (1964); Fiedler (2017)"
    },
    {
      "id": "formalisation",
      "status": "warn",
      "score": 0,
      "weight": 0.05,
      "severity_if_fail": "warning",
      "citation": "Robinaugh et al. (2021); Guest & Martin (2021)"
    },
    {
      "id": "derivation_chain",
      "status": "pass",
      "score": 1,
      "weight": 0.08,
      "severity_if_fail": "blocker",
      "citation": "Scheel et al. (2021); Szollosi et al. (2020)"
    }
  ]
}

The HTML fragment drops straight into a page and renders as a real table:

cat(tf_report(theory, format = "html"))

Rigour report: panic-network

Aggregate score: 87.0 · gate: pass

item	status	score	grounding
falsifiability	pass	1.0	Popper (1959); Bacharach (1989)
precision	pass	1.0	Meehl (1967, 1990)
risk_severity	pass	0.9	Popper (1959); Meehl (1967, 1990)
parsimony	n/a	n/a	Lakatos (1970); Meehl (1990)
non_redundancy	pass	1.0	Kelley (1927); Le et al. (2010); Lawson & Robins (2021)
construct_clarity	pass	1.0	Suddaby (2010); Cronbach & Meehl (1955); Flake & Fried (2020)
scope	pass	1.0	Whetten (1989); Bacharach (1989)
logical_why	pass	1.0	Sutton & Staw (1995); Whetten (1989)
causal_testability	pass	1.0	Textor et al. (2016); Eronen & Bringmann (2021)
diagnosticity	warn	0.0	Platt (1964); Fiedler (2017)
formalisation	warn	0.0	Robinaugh et al. (2021); Guest & Martin (2021)
derivation_chain	pass	1.0	Scheel et al. (2021); Szollosi et al. (2020)

Screening for redundant constructs

The lexical redundancy screen reports the Jaccard similarity and the overlap coefficient of every pair of construct definitions, sorted by descending similarity. A pair is flagged for review when the definitions are near duplicates or one is contained in the other.

tf_redundancy_check(theory)

          a        b similarity overlap flag
1 c_arousal c_threat      0.143    0.25   ok

The lexical screen is deterministic but shallow. When a semantic comparison is wanted, tf_embedding_redundancy() takes any embedder that maps a definition to a numeric vector and reports the cosine similarity of each construct pair. It needs no optional dependency, since the embedder is supplied by the caller. A toy bag-of-words embedder stands in for a real language model here.

vocab <- c("bodily", "activation", "appraised", "danger", "salience")
embedder <- function(def) {
  words <- strsplit(tolower(def), "[^a-z]+")[[1]]
  vapply(vocab, function(w) sum(words == w), numeric(1))
}
tf_embedding_redundancy(theory, embedder)

          a        b   cosine flag
1 c_arousal c_threat 0.408248   ok

Diagramming

tf_diagram() emits deterministic intermediate representations for several diagram types. The digraphs are Graphviz DOT, and the causal DAG uses dagitty syntax.

cat(tf_diagram(theory, type = "nomological_net"))

digraph nomological_net {
  graph [rankdir=LR, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
  node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
  edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
  "c_arousal" [label="Physiological\narousal", fillcolor="#E4F1F1", color="#1E7B7B"];
  "c_threat" [label="Perceived threat", fillcolor="#E4F1F1", color="#1E7B7B"];
  "c_arousal" -> "c_threat" [label="causes"];
}

cat(tf_diagram(theory, type = "causal_dag"))

dag {
  c_arousal -> c_threat
}

The DOT strings render with any Graphviz tool, and tf_render_diagram() does this without leaving R: it returns a DiagrammeR widget for the viewer and for R Markdown, or a standalone SVG string with as = "svg". The packages it uses are optional, in Suggests, so the deterministic core stays dependency-free. Rendered, the nomological net above reads as a figure.

cat(
  '<div class="tf-figure tf-diagram">',
  tf_render_diagram(theory, "nomological_net", as = "svg"),
  '</div>',
  sep = ""
)

The causal DAG is the one view this does not cover, since it emits dagitty syntax rather than DOT. Paste it into a dagitty tool such as dagitty.net instead. tf_implications() reads the same subgraph without leaving R, checking it for acyclicity and returning the conditional independencies it entails, which the Developing and testing article works through.

Simulation

tf_simulate() treats each construct as a state variable and integrates the signed proposition network as a linear dynamical system. The trajectory is deterministic:

sim <- tf_simulate(theory, steps = 5)
unlist(sim$states)

[1] "c_arousal" "c_threat"

unlist(sim$trajectory[[1]]) # the common initial state

[1] 1 1

unlist(sim$trajectory[[6]]) # after five Euler steps

[1] 0.773781 1.181034

The two states separate immediately: arousal, which receives no incoming coupling, decays under the damping term, while perceived threat is pushed up by the positive coupling from arousal before the decay eventually takes over.

The default method = "euler" takes fixed explicit steps, which stay close to the solution only while dt is small against the network’s rates. The function warns when they depart from it by more than 5 per cent. method = "exact" propagates the system with its matrix exponential, so each row is the solution at its time whatever dt is. It will become the default in the next minor release:

exact <- tf_simulate(theory, steps = 5, method = "exact")
unlist(exact$trajectory[[6]]) # the exact solution at time 0.5

[1] 0.778801 1.168201