Retrieve abstracts and richer metadata

Fetches the abstract text and core metadata for one or more records from the Elsevier 'Scopus' Abstract Retrieval API. This complements the Search API used elsewhere in the package: a search returns many records with a few fields each, whereas this returns the fuller record, including the abstract, for a known identifier. Passing include adds the author keywords, the index keywords and the document's reference list, or any of them, to the same request.

Usage

scopus_abstract(
  ids,
  by = c("doi", "scopus_id"),
  view = "META_ABS",
  include = character(),
  cache_dir = NULL,
  resume = TRUE,
  api_key = NULL,
  inst_token = NULL,
  verbose = FALSE
)

Arguments

ids: Character vector of identifiers to look up, either Digital Object Identifiers or 'Scopus' record identifiers (with or without the "SCOPUS_ID:" prefix), according to by.
by: Either "doi" or "scopus_id", the kind of identifier in ids.
view: The Abstract Retrieval view to request: "META_ABS" (the default), "META", "REF" or "FULL", or NULL. "META_ABS" is the view Elsevier documents as carrying the abstract with the core metadata. NULL sends no view parameter, as the 0.4.0 release and earlier did, and leaves the choice to the API. Elsevier documents its default as "META", a view without the abstract, but a live request in October 2026 returned the abstract without a view. Retrieving include = "references" requires view = "FULL" or view = "REF"; see Details for how the two differ and which to prefer.
include: Optional character vector naming extra fields to retrieve in the same request: any of "references", "keywords" (the author keywords) and "index_terms" (the index keywords 'Scopus' assigns). "references" requires view = "FULL" or view = "REF", and "keywords" and "index_terms" require view = "FULL", since the REF response carries neither kind of keyword. Either way this is an entitlement that is separate from ordinary abstract access and from 'Scopus' Search access, and, per Elsevier's own documentation, some fields (notably author keywords) may need to be requested from your Scopus/Elsevier account contact even when the view itself is otherwise accessible. See Details.
cache_dir: Optional directory for per-identifier cache files, as in scopus_fetch_plan(). NULL (the default) performs no caching. Worth setting whenever include is used: Abstract Retrieval draws on its own weekly quota, smaller than and separate from Search's, and every identifier here costs at least one request of its own, so re-running an interrupted batch without a cache re-spends quota already spent. A cache_dir that cannot be written is refused before any request, and a checkpoint that cannot be written all the same warns and keeps the row, as in scopus_fetch_plan().
resume: Logical. When TRUE (the default) and cache_dir is set, an identifier whose cache file already exists is loaded from disk, sparing a second request for it. A cache file that cannot be read back, for example one left half-written by an interrupted run, is retrieved again with a scopus_warning_cache_unreadable warning, and the batch carries on. One holding a different identifier, since distinct identifiers can share a cache filename, is retrieved again with a scopus_warning_cache_mismatch warning.
api_key, inst_token: Optional credentials (see scopus_has_key()).
verbose: Logical. When TRUE, progress is reported.

Value

A tibble of class scopus_abstracts, one row per identifier, with columns id (the input identifier), scopus_id, doi, title, abstract, publication, year and citations. A field the API does not return is NA. An identifier that cannot be retrieved (for example one not in 'Scopus') yields a row of NAs with a warning, so a batch is not lost to a single failure. The number of Abstract Retrieval requests made and the most recently parsed quota (see scopus_quota()) are attached as the n_requests and quota attributes, since this is a materially more expensive operation than a search call.

When include names "keywords", an authkeywords column is added: the document's author-supplied keywords, joined the same way as authors ("; "-separated), or NA when the document has none, or when the API omits the field for a given key's entitlement (see Details).

When include names "index_terms", an idxterms column is added after any authkeywords: the index keywords of the document, joined by "; ", or NA when the response carries none. An entry without text is dropped.

When include names "references", a references list-column is added: one data frame per document, with one row per cited work, where a joined string would have to be parsed apart again. Its columns are position (the reference's place in the bibliography), id (the 'Scopus' identifier of the cited work, when resolved), doi, title, authors, source (the journal or other venue), year and citedbycount (the cited work's own citation count; populated only under view = "REF", NA under "FULL"). A document with no resolvable references yields a zero-row data frame, so the column can always be unnested.

Details

Retrieving references needs Abstract Retrieval's FULL or REF view, and keywords need FULL. FULL returns the whole bibliography in one request. REF serves it a page at a time, about 40 references per request, so scopus_abstract() follows the view's startref parameter until it holds as many references as the document reports, as pybliometrics does. A REF document with N references therefore costs ceiling(N / 40) requests, each counted in n_requests, which makes "FULL" the cheaper view when your entitlement allows it. Elsevier's guide numbers startref from 0 and pybliometrics from 1, and which of the two the API follows has not been confirmed. Paging stops on an empty page, on a page that adds no new reference, or after as many pages as the first page's length implies, so neither numbering can make it loop or repeat references. When the number of references held still differs from the document's reported count, a scopus_warning_incomplete_references warning says so. An identifier whose later page fails is a failed retrieval: it gets a row of NAs with a warning and is not checkpointed.

Author keywords were not populated by either 'Scopus' Search's COMPLETE view (see scopus_records()) or Abstract Retrieval's FULL view in this package's own development testing, against a live, otherwise fully-entitled key, on documents that do carry author keywords in 'Scopus' itself. If your own keywords come back all NA, this is most likely an entitlement gap specific to that field, worth raising with your Scopus/Elsevier account contact. The documents do carry keywords.

Index keywords are the other kind 'Scopus' records. Authors choose their own keywords, whereas 'Scopus' chooses the index keywords and standardises them to vocabularies derived from thesauri Elsevier owns or licenses, with synonyms, spellings and plurals brought together. The two kinds therefore give different keyword co-occurrence results, and a study should say which it used. Elsevier's guide lists index keywords in the FULL view as not available by default, so whether they arrive depends on your key's entitlement as well. In this package's development testing, they did arrive on the same key whose author keywords came back empty.

API access

This performs one request per identifier, or one per page of references under view = "REF" with include = "references", and requires a valid API key and internet access; full-text abstract access, and the FULL/REF views in particular, can also depend on your entitlement. A view or field your key is not entitled to raises a scopus_error_forbidden condition with a message naming the view and suggesting who to contact, where a generic HTTP failure would leave you guessing. Because entitlement is a property of the account, retrieval stops at the first such failure, so the same refusal is not repeated for every remaining identifier.

A spent weekly quota (HTTP 429 marked QUOTA_EXCEEDED) also stops the requests, but returns what the batch holds, since it usually arrives part-way through: the rows already retrieved are kept, each identifier not retrieved gets a row of NAs that is not checkpointed, and one scopus_warning_quota_exceeded warning names the identifier reached, how many were not retrieved and when the quota resets. A checkpoint in cache_dir is still read after the stop. See the API access section of scopus_count() for the other conditions that may be raised.

Examples

if (FALSE) { # scopusflow::scopus_has_key()
# One record of the bundled corpus, looked up for real.
scopus_abstract(scopus_extract_dois(example_records)[1])

# Author keywords and a structured reference list, in the same request.
# Costs one Abstract Retrieval request per identifier, against a smaller,
# separate weekly quota from Search; see the API access section above for
# the entitlement this needs.
rich <- scopus_abstract(
  "10.1038/natrevmats.2016.33",
  view = "FULL", include = c("references", "keywords")
)
rich$references[[1]]
}
# The offline companion, which needs no key. The identifiers, titles,
# sources and citation counts are two records of the bundled corpus of real
# articles; the abstract text is what a live call adds, so it is left as a
# placeholder here, as is the 'Scopus' identifier the corpus does not
# carry.
cited <- example_records[order(-example_records$citations), ][1:2, ]
abstracts <- tibble::tibble(
  id = cited$doi,
  scopus_id = NA_character_,
  doi = cited$doi,
  title = cited$title,
  abstract = NA_character_,
  publication = cited$publication,
  year = cited$year,
  citations = cited$citations
)
class(abstracts) <- c("scopus_abstracts", class(abstracts))
abstracts
#> <scopus_abstracts> (2 records)
#> # A tibble: 2 × 8
#>   id                  scopus_id doi   title abstract publication  year citations
#>   <chr>               <chr>     <chr> <chr> <chr>    <chr>       <int>     <int>
#> 1 10.1038/natrevmats… NA        10.1… Grap… NA       Nature Rev…  2016      1247
#> 2 10.1021/am509065d   NA        10.1… Flex… NA       ACS Applie…  2015       469

# A reference list arrives as one data frame per document, in the
# `references` list-column added by include = "references". The corpus
# carries no bibliographies, so its own records fill the columns here,
# standing in for the works the first document cites.
refs <- example_records[1:3, ]
abstracts$references <- list(
  tibble::tibble(
    position = as.character(seq_len(nrow(refs))),
    id = NA_character_,
    doi = refs$doi,
    title = refs$title,
    authors = refs$authors,
    source = refs$publication,
    year = refs$year,
    citedbycount = refs$citations
  ),
  tibble::tibble()
)
abstracts$references[[1]]
#> # A tibble: 3 × 8
#>   position id    doi                  title    authors source  year citedbycount
#>   <chr>    <chr> <chr>                <chr>    <chr>   <chr>  <int>        <int>
#> 1 1        NA    10.15541/jim20140527 Enhance… Jianhu… Journ…  2015            1
#> 2 2        NA    NA                   Fabrica… Patric… Digit…  2015            0
#> 3 3        NA    10.1021/am509065d    Flexibl… Zhiwei… ACS A…  2015          469