Skip to contents

Mapping the literature

Source: vignettes/articles/literature.Rmd
literature.Rmd

A theory is rarely built in isolation. It sits within a field that already has crowded areas, sparse areas and recurring sets of references. The literature layer of theoryforge gives a deterministic, machine-checkable account of that field, so the placement of a theory within it can be argued from evidence rather than from impression. This article reads a small corpus, computes a bibliometric map, positions a theory against the map, and emits a diagram. The analysis functions are deterministic and require no optional dependency.

Reading a corpus

A corpus is a simple object of the form {schema_version, id, records}, where each record carries an id and may carry title, year, keywords and references. The package ships a small fixture corpus around panic and anxiety research. The format is chosen by file extension, so the same reader handles YAML and JSON.

corpus_path <- system.file("fixtures/panic-corpus.yaml",
                           package = "theoryforge")
corpus <- tf_read_corpus(corpus_path)

corpus$id
[1] "panic-corpus-demo"
length(corpus$records)   # number of works in the corpus
[1] 8
corpus$records[[1]]       # first record
$id
[1] "r1"

$title
[1] "Interoceptive accuracy and panic"

$year
[1] 2018

$keywords
$keywords[[1]]
[1] "arousal"

$keywords[[2]]
[1] "interoception"


$references
$references[[1]]
[1] "clark1986"

$references[[2]]
[1] "barlow2002"

Keywords and references are strings. An unquoted integer, such as a PubMed or Scopus id, is read as its decimal string, so 12345678 and "12345678" name the same work in R and in Python. tf_litmap() and tf_landscape() check the corpus before counting anything, and a message names the record at fault. They stop on a missing or misspelt records key, a record that is not a mapping and a fraction or nested list among the values. They also stop on an integer of 2^53 or more, since R cannot hold one exactly. YAML reads an unquoted NO (nitric oxide), yes, on or off as a logical, so quote such a keyword.

A reference that holds a DOI is keyed by the DOI, lower-cased and without its resolver prefix, because DOI names ignore the case of ASCII letters (DOI Handbook, section 2.4). 10.1016/J.BRAT.2015.10.002 in one record and https://doi.org/10.1016/j.brat.2015.10.002 in another therefore cite one work, which the co-citation map labels 10.1016/j.brat.2015.10.002. Other references, such as Scopus or OpenAlex identifiers, are compared exactly as written, and so are keywords. The package ships the corpus format as a JSON Schema, corpus.schema.json, at system.file("schema", "corpus.schema.json", package = "theoryforge").

Building a literature map

tf_litmap() computes three things from the records. It counts keyword co-occurrence within records, groups co-occurring keywords into thematic components, and counts reference co-citation. An edge is kept only when its co-occurrence count reaches min_link (default 2), which suppresses incidental pairings. Records are read in file order and every result is sorted, so the map is fully deterministic.

lm <- tf_litmap(corpus, min_link = 2)

lm$n_records
[1] 8
unlist(lm$keywords)                 # the vocabulary of the corpus
[1] "appraisal"                      "arousal"                       
[3] "avoidance"                      "catastrophic misinterpretation"
[5] "exposure"                       "genetics"                      
[7] "heritability"                   "interoception"                 
lm$keyword_cooccurrence[[1]]        # first kept co-occurrence edge
$a
[1] "appraisal"

$b
[1] "catastrophic misinterpretation"

$count
[1] 2

The thematic components are the connected groups of co-occurring keywords. The fixture is constructed so that the threshold-two graph breaks into four entirely separate themes.

for (theme in lm$themes) {
  cat(sprintf("%s: %s\n", theme$id,
              paste(unlist(theme$keywords), collapse = ", ")))
}
theme_1: appraisal, catastrophic misinterpretation
theme_2: arousal, interoception
theme_3: avoidance, exposure
theme_4: genetics, heritability

Connected components suit this fixture, built to fall into separate themes. On a real corpus, a few keywords that most records carry, such as the name of the field, link nearly every keyword to every other, so a single component holds the map. Seven OpenAlex corpora of 100 to 600 records each gave a largest theme holding at least 98.8 per cent of the linked keywords. When one theme holds more than half of them, tf_litmap() warns that the themes, and any landscape built on them, are not informative. The map itself is returned unchanged. Below, one keyword shared by two pairs joins them into one theme.

hub <- list(schema_version = "1.0", id = "hub", records = list(
  list(id = "w1", keywords = list("anxiety", "arousal")),
  list(id = "w2", keywords = list("anxiety", "arousal")),
  list(id = "w3", keywords = list("anxiety", "avoidance")),
  list(id = "w4", keywords = list("anxiety", "avoidance"))
))
hub_map <- tf_litmap(hub)
Warning: litmap: one theme holds 100.0 per cent of the 3 linked keywords;
connected components cannot separate themes in a corpus this connected, so the
themes and any landscape built on them are not informative (method
'simple_centres' gives bounded themes)
length(hub_map$themes)
[1] 1

Co-citation is computed the same way over the references field. Two cited sources are linked when at least min_link records in the corpus cite them together.

lm$co_citation
[[1]]
[[1]]$a
[1] "barlow2002"

[[1]]$b
[1] "clark1986"

[[1]]$count
[1] 3


[[2]]
[[2]]$a
[1] "bouton2001"

[[2]]$b
[1] "craske2008"

[[2]]$count
[1] 2

Co-citation maps of real corpora are far larger than keyword maps, because each record cites dozens of works and every pair of them counts. 200 OpenAlex records on panic disorder give 11,211 co-citation edges at the default threshold of two, most of them with a count of exactly two, against 616 keyword edges. min_cocitation sets the co-citation threshold on its own, leaving min_link to the keywords. It defaults to min_link.

tf_litmap(corpus, min_link = 2, min_cocitation = 3)$co_citation
[[1]]
[[1]]$a
[1] "barlow2002"

[[1]]$b
[1] "clark1986"

[[1]]$count
[1] 3

Themes by simple centres

method = "simple_centres" groups keywords as co-word analysis does, by the simple-centres algorithm of Coulter et al. (1998) as Cobo et al. (2011) describe it. Each pair of linked keywords is weighted by its equivalence index: the square of the number of records holding both, divided by the product of the numbers holding each. Two keywords that always occur together score 1. A keyword that most records carry scores little with the rarer keywords it meets, because the index cannot exceed the ratio of the smaller record count to the larger. With the cap on a theme’s size, this keeps a few such keywords from pulling the map into one theme. The strongest link between two keywords not yet in a theme starts one. The theme then takes the free keyword most strongly linked to any of its members, one at a time, until it holds max_theme_size keywords, ten by default, or has no free neighbour. A keyword whose links all reach keywords already in themes joins none, and on the real corpus below about two in five linked keywords end up in no theme. Themes smaller than min_theme_size, two by default, are dropped. max_df leaves out the keywords held by more than that share of the records and lists them as field_terms. At its default of 1, it leaves out none.

On the designed fixture, simple centres find the same four themes as components. The package also ships a real corpus, frozen so that its results stay fixed. It holds the 150 works that OpenAlex ranked first for the search “panic disorder” on 1 October 2026, with OpenAlex’s keywords of that date and without their references. Components put every linked keyword of it in one theme, while simple centres give bounded ones.

openalex <- tf_read_corpus(tf_example_path("openalex-panic-2026.corpus.yaml"))
sizes <- function(lm) vapply(lm$themes, function(th) th$size, integer(1))
sizes(suppressWarnings(tf_litmap(openalex)))
[1] 171
sc <- tf_litmap(openalex, method = "simple_centres")
sizes(sc)
 [1]  5 10 10 10 10  9 10 10  3 10  2  2  2  2  2  2  2
for (theme in sc$themes[1:6]) {
  cat(sprintf("%s %s %s %s | %s\n", theme$id, theme$quadrant, theme$centrality,
              theme$density, paste(unlist(theme$keywords)[1:4], collapse = ", ")))
}
theme_1 basic 2.672471 16.527778 | Anxiety Sensitivity Index, anxiety sensitivity, bodily sensations, body vigilance
theme_2 basic 21.568953 25.107143 | Epidemiologic Catchment Area study, age of onset, alcohol abuse, disability
theme_3 basic 22.628193 20.960784 | Panic Disorder Severity Scale, cognitive behavioral therapy, cognitive restructuring, imipramine
theme_4 motor 8.99724 32.592593 | adverse events, antidepressant efficacy, dose-response relationship, drug safety
theme_5 motor 62.11641 45.993651 | agoraphobia, anxiety disorders, autonomic dysfunction, comorbidities
theme_6 basic 11.797484 10.37037 | alcoholism, behavior therapy, clinical course, depression

Each simple-centres theme also carries the two measures of Callon et al. (1991), scaled as Cobo et al. (2011) scale them. Centrality is ten times the summed index of the theme’s links to the keywords of other themes, and says how strongly the theme is tied to the rest of the field. Density is 100 times the summed index of its internal links, divided by its size, and says how strongly the theme holds together. Splitting each measure at its median places the theme in one quadrant of the strategic diagram. Motor themes are both central and dense, basic themes central but not dense, niche themes dense but peripheral, and the rest "emerging_or_declining". Telling an emerging theme from a declining one needs corpora from more than one period. The result also gains method, the five parameters used and field_terms.

Components remain the default for this release, and the default is planned to become simple centres in the next minor release.

Positioning a theory against the field

tf_landscape() maps a theory’s focal constructs and its registered alternatives onto the themes from tf_litmap(), skipping the co-citation count, which it does not use. A theme is tagged "under_theorised" when neither the focal theory nor any alternative touches it, "covered" when exactly one does, and "crowded" when two or more do. Matching is by shared words between the theme keywords and the construct labels, or the label and key constructs of an alternative, and the result names the words behind every match. Three kinds of word never match. Field tokens, the words most of the corpus shares, are those in the keywords of more than max_token_share of the records, half by default. Phenomenon tokens are the words of the theory’s title, which names the phenomenon that the focal theory and its rivals all explain. A construct word that also appears in the title does not match either. The third kind is a fixed list of words that name a kind of account: theory, model, account, hypothesis, framework and approach, with their plurals. One shared word is enough for a match.

We build a small theory of panic attacks whose constructs speak to arousal and to appraisal, and register one alternative that also speaks to appraisal. The title names the phenomenon, so its words are set aside and the construct labels decide the matches. This leaves the arousal theme covered, the appraisal theme crowded and the remaining themes untouched.

theory <- tf_theory("panic-appraisal", "A theory of panic attacks") |>
  tf_add_construct("c_arousal", "Physiological arousal",
                   "Bodily activation in response to a stressor.") |>
  tf_add_construct("c_appraisal", "Threat appraisal",
                   "Catastrophic appraisal of bodily sensations.") |>
  tf_add_alternative("alt_cog", "Cognitive appraisal account",
                     key_constructs = c("appraisal"))

landscape <- tf_landscape(theory, corpus, min_link = 2)
landscape$theory_id
[1] "panic-appraisal"
unlist(landscape$phenomenon_tokens)
[1] "attacks" "panic"   "theory"

Each theme in the result records which alternatives land on it, whether the focal theory lands on it, the resulting status and the words behind each match, in focal_terms and in alternative_terms.

words <- function(x) if (length(x) > 0L) paste(unlist(x), collapse = ", ") else "none"
for (theme in landscape$themes) {
  alts <- vapply(theme$alternative_terms,
                 function(a) sprintf("%s (%s)", a$id, words(a$terms)), character(1))
  cat(sprintf("%s | status: %s | focal: %s | alternatives: %s\n",
              theme$id, theme$status, words(theme$focal_terms), words(alts)))
}
theme_1 | status: crowded | focal: appraisal | alternatives: alt_cog (appraisal)
theme_2 | status: covered | focal: arousal | alternatives: none
theme_3 | status: under_theorised | focal: none | alternatives: none
theme_4 | status: under_theorised | focal: none | alternatives: none

Two summaries draw out the practical reading. The under-theorised fronts are themes that none of the registered accounts addresses, which says nothing of accounts the theory does not register. The crowded themes, listed under the key redundancy_risk that 0.6.0 gave them, are addressed by two or more accounts, so they call for predictions that discriminate between those accounts. They do not show that any account is redundant.

unlist(landscape$under_theorised_fronts)
[1] "theme_3" "theme_4"
unlist(landscape$redundancy_risk)
[1] "theme_1"

method chooses how tf_litmap() builds the themes. With "simple_centres", the result gains method after theory_id, and each theme keeps its centrality, density and quadrant. A front that none of the accounts addresses can then be read with its place in the strategic diagram.

real <- tf_landscape(theory, openalex, method = "simple_centres")
for (theme in real$themes[1:6]) {
  cat(sprintf("%s %s %s | %s\n", theme$id, theme$status, theme$quadrant,
              paste(unlist(theme$keywords)[1:3], collapse = ", ")))
}
theme_1 under_theorised basic | Anxiety Sensitivity Index, anxiety sensitivity, bodily sensations
theme_2 under_theorised basic | Epidemiologic Catchment Area study, age of onset, alcohol abuse
theme_3 covered basic | Panic Disorder Severity Scale, cognitive behavioral therapy, cognitive restructuring
theme_4 under_theorised motor | adverse events, antidepressant efficacy, dose-response relationship
theme_5 under_theorised motor | agoraphobia, anxiety disorders, autonomic dysfunction
theme_6 under_theorised basic | alcoholism, behavior therapy, clinical course

Emitting a literature diagram

tf_lit_diagram() renders a deterministic Graphviz DOT string for the literature layer. The keyword co-occurrence and co-citation maps render as undirected graphs, and a landscape result renders as a directed theme map. The output is a plain string, so it can be written to a .dot file or passed to a Graphviz renderer.

cat(tf_lit_diagram(lm, type = "keyword_cooccurrence"))
graph keyword_cooccurrence {
  graph [rankdir=LR, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
  node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
  edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
  node [shape=ellipse, style="filled", fillcolor="#E4F1F1", color="#1E7B7B"];
  "appraisal";
  "arousal";
  "avoidance";
  "catastrophic misinterpretation";
  "exposure";
  "genetics";
  "heritability";
  "interoception";
  "appraisal" -- "catastrophic misinterpretation" [label="2"];
  "arousal" -- "interoception" [label="2"];
  "avoidance" -- "exposure" [label="2"];
  "genetics" -- "heritability" [label="2"];
}

tf_render_diagram() accepts a DOT string as well as a theory, so the literature diagrams render the same way as the theory views.

cat('<div class="tf-figure tf-diagram">',
    tf_render_diagram(
      tf_lit_diagram(lm, type = "keyword_cooccurrence"),
      as = "svg"
    ),
    '</div>',
    sep = "")
keyword_cooccurrence appraisal appraisal catastrophic misinterpretation catastrophic misinterpretation appraisal--catastrophic misinterpretation 2 arousal arousal interoception interoception arousal--interoception 2 avoidance avoidance exposure exposure avoidance--exposure 2 genetics genetics heritability heritability genetics--heritability 2

The co-citation map links the references that the corpus cites together, so tightly co-cited work stands out as the intellectual base the field shares.

cat(tf_lit_diagram(lm, type = "co_citation"))
graph co_citation {
  graph [rankdir=LR, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
  node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
  edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
  node [shape=ellipse, style="filled", fillcolor="#E7EDF5", color="#33567A"];
  "barlow2002";
  "bouton2001";
  "clark1986";
  "craske2008";
  "barlow2002" -- "clark1986" [label="3"];
  "bouton2001" -- "craske2008" [label="2"];
}
cat('<div class="tf-figure tf-diagram">',
    tf_render_diagram(
      tf_lit_diagram(lm, type = "co_citation"),
      as = "svg"
    ),
    '</div>',
    sep = "")
co_citation barlow2002 barlow2002 clark1986 clark1986 barlow2002--clark1986 3 bouton2001 bouton2001 craske2008 craske2008 bouton2001--craske2008 2

On a real corpus that diagram can run to thousands of edges and a megabyte of DOT, which Graphviz cannot lay out legibly. max_edges keeps the strongest edges, the highest counts first with ties going to the pair that sorts first, and draws only their endpoints. It applies to both network views.

cat(tf_lit_diagram(lm, type = "co_citation", max_edges = 1))
graph co_citation {
  graph [rankdir=LR, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
  node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
  edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
  node [shape=ellipse, style="filled", fillcolor="#E7EDF5", color="#33567A"];
  "barlow2002";
  "clark1986";
  "barlow2002" -- "clark1986" [label="3"];
}

The theme landscape diagram is produced from the tf_landscape() result rather than from the map, since it carries the focal theory and the alternatives.

cat(tf_lit_diagram(landscape, type = "theme_landscape"))
digraph theme_landscape {
  graph [rankdir=LR, bgcolor="transparent", fontname="Helvetica", fontsize=11, pad="0.2", nodesep="0.3", ranksep="0.45"];
  node [fontname="Helvetica", fontsize=11, shape=box, style="rounded,filled", color="#33567A", fillcolor="#F2F6F9", fontcolor="#12283A", penwidth=1.1, margin="0.16,0.1"];
  edge [fontname="Helvetica", fontsize=10, color="#7B909F", fontcolor="#0F6E6E", arrowsize=0.7];
  "theme_1" [label="theme_1\nappraisal, catastrophic\nmisinterpretation\n(crowded)", fillcolor="#FBF1DC", color="#9C6B14"];
  "theme_2" [label="theme_2\narousal, interoception\n(covered)", fillcolor="#F1F1F1", color="#8A8A8A"];
  "theme_3" [label="theme_3\navoidance, exposure\n(under_theorised)", fillcolor="#E4F1F1", color="#1E7B7B"];
  "theme_4" [label="theme_4\ngenetics, heritability\n(under_theorised)", fillcolor="#E4F1F1", color="#1E7B7B"];
  "alt_cog" [label="alt_cog", shape=ellipse, fillcolor="#F1F1F1", color="#8A8A8A"];
  "focal" [label="focal", shape=ellipse, fillcolor="#12283A", color="#12283A", fontcolor="#FFFFFF"];
  "alt_cog" -> "theme_1";
  "focal" -> "theme_1";
  "focal" -> "theme_2";
}
cat('<div class="tf-figure tf-diagram">',
    tf_render_diagram(
      tf_lit_diagram(landscape, type = "theme_landscape"),
      as = "svg"
    ),
    '</div>',
    sep = "")
theme_landscape theme_1 theme_1appraisal, catastrophicmisinterpretation(crowded) theme_2 theme_2arousal, interoception(covered) theme_3 theme_3avoidance, exposure(under_theorised) theme_4 theme_4genetics, heritability(under_theorised) alt_cog alt_cog alt_cog->theme_1 focal focal focal->theme_1 focal->theme_2

Fetching a corpus from OpenAlex

The corpus above was read from a file. A corpus can also be assembled from the OpenAlex works API with tf_fetch_corpus(). This adapter is assistive. It makes a live network call and its result depends on an external service that changes over time, so it sits outside the deterministic core of the package. The call is shown here but not run.

fetched <- tf_fetch_corpus("panic disorder interoception",
                           per_page = 100,
                           max_records = 400)
fetched$source    # the query, when it ran, how many works matched and were kept
jsonlite::write_json(fetched, "panic-corpus.json", auto_unbox = TRUE,
                     null = "null")
fetched <- tf_read_corpus("panic-corpus.json")
fetched_map <- tf_litmap(fetched)

OpenAlex answers requests without a key within a budget of USD 0.10 a day. A search page costs USD 0.001, so the budget covers about 100 pages. A free API key raises the budget to USD 1 a day. Store the key in the OPENALEX_API_KEY environment variable, for example in .Renviron, and tf_fetch_corpus() sends it in a request header, never in the URL or the corpus. The mailto argument is still accepted, but OpenAlex now ignores it. When OpenAlex refuses a request, typically with HTTP 429 once the budget is spent, the function stops with the status and OpenAlex’s message. It used to return an empty corpus, which read as a literature with no themes.

Each request returns one page of works, ranked by relevance, from a result set that is usually far larger: a search for panic disorder matched about 187,000 works in October 2026. OpenAlex supports pages of up to 100 works and has deprecated larger ones, although it still accepts 200. max_records pages on through OpenAlex’s cursor until that many works are collected, and fetched$source records the query, the UTC time of retrieval, the number of works that matched and the number kept. That record matters because the keywords change. Since late September 2026, OpenAlex has written each work’s keywords with a language model that reads its title, abstract and venue. It merges and splits that vocabulary over time, so the same works fetched a week apart can carry different keywords. Works left without keywords fall back to their concepts, a deprecated vocabulary whose capitalised names do not match the lower-case keywords. Each record also keeps the work’s DOI as OpenAlex gives it. The example writes the corpus with null = "null", because jsonlite would otherwise write a missing DOI or count as an empty object, which the corpus schema does not allow.

Once fetched, the corpus object has the same shape as a file-read corpus, so it flows into tf_litmap(), tf_landscape() and tf_lit_diagram() without any further change. The file-read corpus used throughout this article shows that shape.

str(corpus, max.level = 2, list.len = 4)
List of 3
 $ schema_version: chr "1.0"
 $ id            : chr "panic-corpus-demo"
 $ records       :List of 8
  ..$ :List of 5
  ..$ :List of 5
  ..$ :List of 5
  ..$ :List of 5
  .. [list output truncated]

The recommended pattern is to fetch once, write the result to disk, and then work from the saved file so that later analyses remain reproducible.

Locating a corpus is only one use of a literature search. A second, recurring need is narrower: checking whether a search has turned up any source the theory does not already cite. tf_new_evidence_dois() answers that question deterministically, from a theory object and a plain vector of candidate DOIs, regardless of where the DOIs came from.

The bundled panic theory records one DOI as evidence and one for each of its two registered alternatives, and a DOI cited as an alternative counts as cited just as an evidence DOI does. Those are the sources the candidate list below is checked against.

panic <- tf_read(system.file("fixtures", "panic-network.theory.yaml",
                             package = "theoryforge"))

candidates <- c(
  "10.1016/j.brat.2015.10.002",                   # already cited as evidence
  # already cited as an alternative, in URL form
  "https://doi.org/10.1016/0005-7967(86)90011-2",
  "https://doi.org/10.1037/0033-2909.99.1.20"     # not yet cited
)

tf_new_evidence_dois(panic, candidates)
[1] "https://doi.org/10.1037/0033-2909.99.1.20"

The comparison is on a normalised form of each DOI, the DOI itself found wherever it sits in the text, lowercased and percent-decoded. A plain DOI, a resolvable URL and the forms doi: 10... and DOI 10... for the same work are therefore recognised as the same source, and a trailing full stop is ignored. The function takes no network dependency itself: the search is left entirely to whichever tool supplies the candidate list.

Combining it with scopusflow

scopusflow is a companion R package, by the same author, for querying the Elsevier Scopus Search API. It is a natural source of candidate DOIs: scopus_fetch() retrieves records for a query, and scopus_extract_dois() reduces them to a plain DOI vector, which is exactly the input tf_new_evidence_dois() expects.

scopusflow is not a dependency of theoryforge, so the snippet below is illustrative rather than executed by this vignette. Install scopusflow and supply a Scopus API key (see its scopus_has_key()) to run it.

library(scopusflow)

query <- scopus_query("panic disorder", "interoception", .op = "AND")
records <- scopus_fetch(query, years = 2015:2026)
candidates <- scopus_extract_dois(records)

tf_new_evidence_dois(panic, candidates)

scopus_diff_dois() extends this to tracking a search over time: it compares an earlier and a later retrieval (either scopus_records objects or DOI vectors) and reports which DOIs were added, removed or unchanged. Re-running a saved query and passing the "added" DOIs into tf_new_evidence_dois() gives a routine for revisiting a theory’s evidence base as the literature grows.

records_2025 <- scopus_fetch(query, years = 2015:2025)
records_2026 <- scopus_fetch(query, years = 2015:2026)
diff <- scopus_diff_dois(records_2025, records_2026)
added <- diff$doi[diff$status == "added"]

tf_new_evidence_dois(panic, added)

The hand-off itself needs nothing from scopusflow: scopus_diff_dois() returns one row per DOI with its status, which we reproduce here so the last step runs. Only the newly added DOI comes back as new evidence.

diff <- data.frame(
  doi = c("10.1016/j.brat.2015.10.002", "10.1037/0033-2909.99.1.20"),
  status = c("unchanged", "added")
)
added <- diff$doi[diff$status == "added"]

tf_new_evidence_dois(panic, added)
[1] "10.1037/0033-2909.99.1.20"

scopusflow also offers scopus_compare_topics(), which tracks the relative publication share of several comparison terms against a reference term across a range of years. This suits comparing a theory against its registered alternatives on their standing in the literature, using the alternatives’ labels as the comparison terms.

scopus_compare_topics(
  reference_query = "panic disorder",
  comparison_terms = c("cognitive appraisal account", "biological account"),
  years = 2015:2026
)

The comparison and its plot, with per-year stability bands, are shown in scopusflow’s Comparing topics article.

Using scopusflow for a Scopus-based corpus

tf_litmap() and tf_landscape() read the keywords and references fields of each corpus record, and scopusflow supplies both. scopus_corpus() takes the records from scopus_fetch() and enriches them, through Abstract Retrieval, into a tibble of id, title, year, keywords (one character vector of author keywords per record) and references (one data frame of cited works per record, with id, doi, title and other fields).

tf_fetch_corpus() (OpenAlex) stays the built-in default because OpenAlex is free and answers modest use without a key, so the literature layer works with no setup. Scopus needs an institutional subscription and an API key, so scopusflow is an opt-in source rather than a dependency. The two packages exchange plain data, a DOI vector or a corpus written to a file, with no coupling in either direction. That keeps theoryforge dependency-light and usable straight after install, and lets a reader reach for whichever index they have access to.

The corpus format expects a top-level {schema_version, id, records} mapping and, within each record, references as a flat list of id strings, whereas scopus_corpus() returns a tibble whose references entries are data frames. Fetch the records and enrich them first:

# illustrative: needs scopusflow and a configured Scopus API key
recs <- scopusflow::scopus_fetch(query, years = 2015:2026)
# id, title, year, keywords, references
sc   <- scopusflow::scopus_corpus(recs)

tf_litmap() folds the case of a DOI and drops its resolver prefix, but it compares every other reference and every keyword as an exact string. The conversion therefore has to give each cited work one key across the whole corpus. Scopus gives a cited work’s DOI in one record and not at all in another. A key built from the DOI where there is one, and from the Scopus identifier otherwise, splits one work into two nodes, and the co-citation map loses its edges. Author keywords keep their authors’ capitals, which splits the keyword themes in the same way. The adapter below therefore keys each cited work by its Scopus identifier, as scopus:<id>. A reference that carries only a DOI borrows the identifier of another reference with the same DOI anywhere in the corpus, and is keyed as doi:<doi>, with the DOI lower-cased and any resolver prefix removed, when no such reference exists. tf_litmap() labels such a work with the bare DOI. References with neither are dropped. Keywords are trimmed and lower-cased, and a missing year is written as null.

scopus_corpus_to_tf <- function(sc, id) {
  # DOIs are case-insensitive over ASCII letters only (DOI Handbook, 2.4), so
  # the fold leaves every other character alone.
  fold_doi <- function(doi) {
    doi <- chartr(paste(LETTERS, collapse = ""), paste(letters, collapse = ""),
                  trimws(doi))
    sub("^(https?://(dx\\.)?doi\\.org/|doi:)", "", doi)
  }
  present <- function(x) {
    x <- trimws(as.character(x))
    x[!is.na(x) & !nzchar(x)] <- NA_character_
    x
  }
  # One (Scopus id, DOI) pair per cited work. A record whose lookup failed or
  # found no references has an empty entry.
  cited <- lapply(sc$references, function(refs) {
    if (!is.data.frame(refs) || nrow(refs) == 0L) {
      return(data.frame(sid = character(), doi = character()))
    }
    data.frame(sid = present(refs$id), doi = fold_doi(present(refs$doi)))
  })
  # Every reference that carries both says which Scopus id a DOI belongs to.
  pairs <- do.call(rbind, c(list(data.frame(sid = character(),
                                            doi = character())), cited))
  pairs <- pairs[!is.na(pairs$sid) & !is.na(pairs$doi), , drop = FALSE]
  pairs <- pairs[!duplicated(pairs$doi), , drop = FALSE]
  sid_of_doi <- stats::setNames(pairs$sid, pairs$doi)

  records <- lapply(seq_len(nrow(sc)), function(i) {
    refs <- cited[[i]]
    sid <- refs$sid
    borrow <- is.na(sid) & refs$doi %in% names(sid_of_doi)
    sid[borrow] <- sid_of_doi[refs$doi[borrow]]
    keys <- ifelse(!is.na(sid), paste0("scopus:", sid),
                   ifelse(!is.na(refs$doi), paste0("doi:", refs$doi), NA))
    keywords <- tolower(present(sc$keywords[[i]]))
    record <- list(id = as.character(sc$id[[i]]),
                   title = present(sc$title[[i]]),
                   year = as.integer(sc$year[[i]]),
                   keywords = I(unique(keywords[!is.na(keywords)])),
                   references = I(unique(keys[!is.na(keys)])))
    # The schema accepts a null year, but a title is a string.
    if (is.na(record$title)) record$title <- NULL
    record
  })
  list(schema_version = "1.0", id = id, records = records)
}

Write the result with null = "null" and na = "null", so that a missing year reaches the file as null (by default jsonlite writes the string "NA"), then read it back with tf_read_corpus(). I() in the adapter keeps a one-element keyword or reference list an array under auto_unbox = TRUE.

# illustrative: continues from the fetch above
lit <- scopus_corpus_to_tf(sc, "scopus:panic disorder")
jsonlite::write_json(lit, "corpus.json", auto_unbox = TRUE,
                     null = "null", na = "null")

corpus <- tf_read_corpus("corpus.json")
tf_litmap(corpus)

The next release of scopusflow is planned to write this file itself, with the same keys, so that the adapter is then needed only for earlier versions.