scopusflow: A literature search you can rerun

Describe the search first

The bundled example follows research on graphene supercapacitors. Query helpers add the field syntax, while scopus_plan() divides the work into yearly cells. The resulting object can be inspected and saved before any quota is spent.

query <- scopus_query('graphene supercapacitor', .field = 'TITLE-ABS-KEY')
plan <- scopus_plan(query, years = 2015:2024, partition = 'year')
query

#> [1] "TITLE-ABS-KEY(graphene supercapacitor)"

plan

#> <scopus_plan> (10 cells, view "STANDARD", partition "year")
#> # A tibble: 10 × 6
#>     cell query                                  date   year view     page_size
#>  * <int> <chr>                                  <chr> <int> <chr>        <int>
#>  1     1 TITLE-ABS-KEY(graphene supercapacitor) 2015   2015 STANDARD       200
#>  2     2 TITLE-ABS-KEY(graphene supercapacitor) 2016   2016 STANDARD       200
#>  3     3 TITLE-ABS-KEY(graphene supercapacitor) 2017   2017 STANDARD       200
#>  4     4 TITLE-ABS-KEY(graphene supercapacitor) 2018   2018 STANDARD       200
#>  5     5 TITLE-ABS-KEY(graphene supercapacitor) 2019   2019 STANDARD       200
#>  6     6 TITLE-ABS-KEY(graphene supercapacitor) 2020   2020 STANDARD       200
#>  7     7 TITLE-ABS-KEY(graphene supercapacitor) 2021   2021 STANDARD       200
#>  8     8 TITLE-ABS-KEY(graphene supercapacitor) 2022   2022 STANDARD       200
#>  9     9 TITLE-ABS-KEY(graphene supercapacitor) 2023   2023 STANDARD       200
#> 10    10 TITLE-ABS-KEY(graphene supercapacitor) 2024   2024 STANDARD       200

The plan already supports a draft methods description. Any details unavailable at this stage are marked as unrecorded. The report follows PRISMA-S, the reporting extension for literature searches (Rethlefsen et al., 2021).

cat(format(scopus_search_report(plan), style = 'paragraph'))

#> The search described here has not been run. The search expression is TITLE-ABS-KEY(graphene supercapacitor), limited to publication years 2015 to 2024. It would be partitioned into 10 cells, one per year, each retrieved through the STANDARD view in pages of 200 records, under a paging mode chosen when the search is run. The PRISMA-S items this record cannot supply, among them peer review of the strategy, grey literature and any other database searched, remain yours to report.

Resume from the last completed cell

The bundled corpus now stands in for the result of that same plan. The cache is deliberately empty at the start of the example.

source('stand_in.R')
stand_in <- scopus_stand_in(example_records)
options(scopusflow.api_key = 'offline-demo-key')

cache <- file.path(tempdir(), 'scopusflow-blog-cache')
unlink(cache, recursive = TRUE)

httr2::with_mocked_responses(stand_in$handler, {
  records <- scopus_fetch_plan(plan, cache_dir = cache, resume = TRUE)
})
c(records = nrow(records), requests = stand_in$state$requests)

#>  records requests 
#>      138       10

Each year is written to its own checkpoint. Repeating the plan reloads those files and makes no further request.

requests_before <- stand_in$state$requests
httr2::with_mocked_responses(stand_in$handler, {
  repeated <- scopus_fetch_plan(plan, cache_dir = cache, resume = TRUE)
})
c(records = nrow(repeated),
  new_requests = stand_in$state$requests - requests_before)

#>      records new_requests 
#>          138            0

The practical benefit is clearest after an interruption. Work already completed is retained, while the query, years, view and page size stored beside each checkpoint prevent an incompatible plan from silently reusing it.

Inspect the result

The records retain their plan, retrieval time, package version and count for each cell. That provenance is enough to plot the development of the bundled literature and later distinguish a changed search from a changed method.

httr2::with_mocked_responses(stand_in$handler, {
  trend <- scopus_trend('graphene supercapacitor', years = 2015:2024,
                        field = 'TITLE-ABS-KEY')
})
plot_scopus_trend(trend) +
  ggplot2::labs(
    title = 'Records in the demonstration corpus',
    caption = 'Offline OpenAlex-derived demonstration records, 2015-2024'
  ) +
  ggplot2::theme(
    plot.title = ggplot2::element_text(size = 11, face = 'bold'),
    axis.title = ggplot2::element_text(size = 10),
    axis.text = ggplot2::element_text(size = 9),
    plot.caption = ggplot2::element_text(size = 8.5, colour = 'grey35')
  )

An update can be compared by DOI. In this illustration, the baseline ends in 2023 and the later harvest includes the 2024 records.

baseline <- records[records$year <= 2023, ]
changes <- scopus_diff_dois(old = baseline, new = records)
changes

#> <scopus_doi_diff> 14 added, 0 removed, 113 unchanged
#> # A tibble: 127 × 2
#>    doi                            status
#>    <chr>                          <fct> 
#>  1 10.1002/adfm.202315137         added 
#>  2 10.1002/asia.202400548         added 
#>  3 10.1002/slct.202302535         added 
#>  4 10.1016/j.cej.2024.148822      added 
#>  5 10.1016/j.diamond.2024.110842  added 
#>  6 10.1016/j.isci.2024.111696     added 
#>  7 10.1016/j.jallcom.2024.175000  added 
#>  8 10.1016/j.jallcom.2024.177248  added 
#>  9 10.1016/j.jpowsour.2024.234127 added 
#> 10 10.1016/j.jpowsour.2024.236149 added 
#> # ℹ 117 more rows

Let the result write the record

The search record is generated from the object that contains the publications. It reports what the software knows and identifies the PRISMA-S items that still require the review author’s judgement, such as citation searching or peer review of the strategy. Because this run used a stand-in rather than Scopus, the compact summary below omits the database, platform and retrieval date that a live report would supply.

The same records can then leave R as a DOI list, BibTeX, RIS or a table for bibliometrix. Saving the native object preserves the attached provenance for a future update.

Limits

The offline corpus demonstrates the workflow. Its results do not represent the contents of Scopus. A live search still depends on Elsevier’s coverage, indexing and quota, and abstract retrieval uses a separate allowance. Scopus records also cannot be redistributed, which is why this article uses records obtained from OpenAlex and reshaped to the package’s return format.

The R reference and Python reference cover cursor paging, de-duplication, topic comparisons and live authentication. For a review, I would keep the plan under version control, save the native result object and archive the generated search record with the screening data.