Retrieve abstracts and richer metadata

Fetches the abstract text and core metadata for one or more records from the Elsevier 'Scopus' Abstract Retrieval API. This complements the Search API used elsewhere in the package: a search returns many records with a few fields each, whereas this returns the fuller record, including the abstract, for a known identifier. Passing include adds author keywords and/or the document's reference list to the same request.

Usage

scopus_abstract(
  ids,
  by = c("doi", "scopus_id"),
  view = NULL,
  include = character(),
  cache_dir = NULL,
  resume = TRUE,
  api_key = NULL,
  inst_token = NULL,
  verbose = FALSE
)

Arguments

ids: Character vector of identifiers to look up, either Digital Object Identifiers or 'Scopus' record identifiers (with or without the "SCOPUS_ID:" prefix), according to by.
by: Either "doi" or "scopus_id", the kind of identifier in ids.
view: Optional character scalar naming the Abstract Retrieval view to request: one of "META", "META_ABS", "REF" or "FULL". NULL (the default) omits the view parameter from the request entirely, exactly as scopusflow has always done, so existing calls that never mention view are unaffected. Retrieving include = "references" requires view = "FULL" or view = "REF"; see Details for how the two differ and which to prefer.
include: Optional character vector naming extra fields to retrieve in the same request: "references" and/or "keywords". "references" requires view = "FULL" or view = "REF", and "keywords" requires view = "FULL", since the REF response carries no author keywords. Either way this is an entitlement that is separate from ordinary abstract access and from 'Scopus' Search access, and, per Elsevier's own documentation, some fields (notably author keywords) may need to be requested from your Scopus/Elsevier account contact even when the view itself is otherwise accessible. See Details.
cache_dir: Optional directory for per-identifier cache files, as in scopus_fetch_plan(). NULL (the default) performs no caching. Worth setting whenever include is used: Abstract Retrieval draws on its own weekly quota, smaller than and separate from Search's, and every identifier here costs its own request, so re-running an interrupted batch without a cache re-spends quota already spent.
resume: Logical. When TRUE (the default) and cache_dir is set, an identifier whose cache file already exists is loaded from disk rather than requested again.
api_key, inst_token: Optional credentials (see scopus_has_key()).
verbose: Logical. When TRUE, progress is reported.

Value

A tibble of class scopus_abstracts, one row per identifier, with columns id (the input identifier), scopus_id, doi, title, abstract, publication, year and citations. A field the API does not return is NA. An identifier that cannot be retrieved (for example one not in 'Scopus') yields a row of NAs with a warning, so a batch is not lost to a single failure. The number of Abstract Retrieval requests made and the most recently parsed quota (see scopus_quota()) are attached as the n_requests and quota attributes, since this is a materially more expensive operation than a search call.

When include names "keywords", an authkeywords column is added: the document's author-supplied keywords, joined the same way as authors ("; "-separated), or NA when the document has none, or when the API omits the field for a given key's entitlement (see Details).

When include names "references", a references list-column is added: one data frame per document, with one row per cited work, rather than a single joined string. Its columns are position (the reference's place in the bibliography), id (the 'Scopus' identifier of the cited work, when resolved), doi, title, authors, source (the journal or other venue), year and citedbycount (the cited work's own citation count; populated only under view = "REF", NA under "FULL"). A document with no resolvable references yields a zero-row data frame, not NA, so the column can always be unnested.

Details

Retrieving references needs Abstract Retrieval's FULL or REF view, and keywords need FULL. In development, against a live key with full Abstract Retrieval access, view = "FULL" returned a complete, correctly counted reference list for every document tried. view = "REF" returned the identical, complete list in one case but a truncated (paginated) subset in another, on an otherwise identical request made moments apart, so "FULL" is recommended when your entitlement allows it. "REF" remains available for accounts entitled only to it; when the number of references returned does not match the document's own reported reference count, a warning is issued naming the identifier, since the list may be an incomplete page rather than the whole bibliography.

Author keywords were not populated by either 'Scopus' Search's COMPLETE view (see scopus_records()) or Abstract Retrieval's FULL view in this package's own development testing, against a live, otherwise fully-entitled key, on documents that do carry author keywords in 'Scopus' itself. If your own keywords come back all NA, this is most likely an entitlement gap specific to that field, worth raising with your Scopus/Elsevier account contact, rather than the documents genuinely having none.

API access

This performs one request per identifier and requires a valid API key and internet access; full-text abstract access, and the FULL/REF views in particular, can also depend on your entitlement. A view or field your key is not entitled to raises a scopus_error_forbidden condition with a message naming the view and suggesting who to contact, rather than a generic HTTP failure; because entitlement is an account-level property, not a per-document one, retrieval stops at the first such failure instead of repeating it for every remaining identifier. See the API access section of scopus_count() for the other conditions that may be raised.

Examples

if (FALSE) { # scopusflow::scopus_has_key()
# One record of the bundled corpus, looked up for real.
scopus_abstract(scopus_extract_dois(example_records)[1])

# Author keywords and a structured reference list, in the same request.
# Costs one Abstract Retrieval request per identifier, against a smaller,
# separate weekly quota from Search; see the API access section above for
# the entitlement this needs.
rich <- scopus_abstract(
  "10.1038/natrevmats.2016.33",
  view = "FULL", include = c("references", "keywords")
)
rich$references[[1]]
}
# The offline companion, which needs no key. The identifiers, titles,
# sources and citation counts are two records of the bundled corpus of real
# articles; the abstract text is what a live call adds, so it is left unset
# here rather than invented, as is the 'Scopus' identifier the corpus does
# not carry.
cited <- example_records[order(-example_records$citations), ][1:2, ]
abstracts <- tibble::tibble(
  id = cited$doi,
  scopus_id = NA_character_,
  doi = cited$doi,
  title = cited$title,
  abstract = NA_character_,
  publication = cited$publication,
  year = cited$year,
  citations = cited$citations
)
class(abstracts) <- c("scopus_abstracts", class(abstracts))
abstracts
#> <scopus_abstracts> (2 records)
#> # A tibble: 2 × 8
#>   id                  scopus_id doi   title abstract publication  year citations
#>   <chr>               <chr>     <chr> <chr> <chr>    <chr>       <int>     <int>
#> 1 10.1038/natrevmats… NA        10.1… Grap… NA       Nature Rev…  2016      1247
#> 2 10.1021/am509065d   NA        10.1… Flex… NA       ACS Applie…  2015       469

# A reference list arrives as one data frame per document, in the
# `references` list-column added by include = "references". The corpus
# carries no bibliographies, so its own records fill the columns here,
# standing in for the works the first document cites.
refs <- example_records[1:3, ]
abstracts$references <- list(
  tibble::tibble(
    position = as.character(seq_len(nrow(refs))),
    id = NA_character_,
    doi = refs$doi,
    title = refs$title,
    authors = refs$authors,
    source = refs$publication,
    year = refs$year,
    citedbycount = refs$citations
  ),
  tibble::tibble()
)
abstracts$references[[1]]
#> # A tibble: 3 × 8
#>   position id    doi                  title    authors source  year citedbycount
#>   <chr>    <chr> <chr>                <chr>    <chr>   <chr>  <int>        <int>
#> 1 1        NA    10.15541/jim20140527 Enhance… Jianhu… Journ…  2015            1
#> 2 2        NA    NA                   Fabrica… Patric… Digit…  2015            0
#> 3 3        NA    10.1021/am509065d    Flexibl… Zhiwei… ACS A…  2015          469