Who is through with convergence warnings? Publications that used allFit to compare optimisers

Where a function name can be found

A function name is a poor search term for bibliographic databases, because it appears in the methods section of an article and almost never in its title, abstract or keywords. Three sources with different coverage are therefore searched, by a script in the website’s repository that a GitHub Actions workflow runs on request.

Scopus, searched through the scopusflow package, indexes titles, abstracts, keywords, references and a few other fields, but not the body of the article, so a search for the function name in any indexed field finds only articles that mention it in one of those. A second Scopus query, drafted for the first version of this post in 2023, is a proxy: it looks for abstracts that mention lme4, lmerTest or brms alongside maximal random slopes and convergence, on the grounds that an abstract discussing convergence at that level of detail is likely to belong to a study that dealt with it. Europe PMC indexes the full text of a large body of life-science articles, most of them open access but not all, and its search interface accepts a full-text query. OpenAlex offers a full-text search over the articles for which it holds the text. Both full-text queries pair the function name with a mixed-model term, which removes some unrelated uses of the string without removing all of them.

library(scopusflow)   # reads the key from SCOPUS_API_KEY
scopus_any_field <- scopus_fetch('ALL("allFit")')
scopus_proxy <- scopus_fetch(
  paste('("lme4" OR "lmerTest" OR "brms") AND maximal AND "random slopes"',
        'AND (convergence OR converge OR converged OR converging)'),
  field = 'TITLE-ABS-KEY')

The two full-text queries are shown as the workflow recorded them, so that what appears here is what actually ran.

cat(readLines('searches/europepmc_query.txt'), sep = '\n')

"allFit" AND (lme4 OR "mixed-effects" OR "mixed effects" OR "mixed model" OR "mixed models" OR "multilevel")

cat(readLines('searches/openalex_filter.txt'), sep = '\n')

fulltext.search:allFit AND (lme4 OR lmer OR "mixed effects" OR "mixed-effects" OR "mixed model" OR "mixed models" OR multilevel)

The list

The workflow commits its results to this post’s directory, and the tables below are built from those files, so the record can be refreshed without editing the post.

library(dplyr)
library(ggplot2)

read_if_present <- function(file, source) {
  path <- file.path('searches', file)
  if (!file.exists(path)) return(NULL)
  x <- read.csv(path, stringsAsFactors = FALSE)
  x$source <- source
  x
}
retrieved <- readLines('searches/retrieved.txt')

scopus_any   <- read_if_present('scopus_allfit_any_field.csv', 'Scopus, any indexed field')
scopus_proxy <- read_if_present('scopus_convergence_proxy.csv', 'Scopus, convergence proxy')
europepmc    <- read_if_present('europepmc_allfit_fulltext.csv', 'Europe PMC, full text')
openalex     <- read_if_present('openalex_allfit_fulltext.csv', 'OpenAlex, full text')

standardise <- function(x) {
  if (is.null(x)) return(NULL)
  tibble(source = x$source,
         doi = tolower(sub('^https?://doi.org/', '', x$doi)),
         year = as.integer(x$year),
         authors = x$authors,
         title = x$title,
         venue = if ('journal' %in% names(x)) x$journal else x$publication)
}
records <- bind_rows(lapply(list(scopus_any, scopus_proxy, europepmc, openalex),
                            standardise))
records |> count(source, name = 'records')

#> # A tibble: 4 × 2
#>   source                    records
#>   <chr>                       <int>
#> 1 Europe PMC, full text          30
#> 2 OpenAlex, full text            68
#> 3 Scopus, any indexed field     176
#> 4 Scopus, convergence proxy       1

# Do any of the Scopus any-field titles mention a mixed model at all?
mixed_terms <- 'mixed|lme4|multilevel|random effect|convergence|optimi[sz]'
sum(grepl(mixed_terms, scopus_any$title, ignore.case = TRUE))

#> [1] 0

head(sort(table(scopus_any$publication), decreasing = TRUE), 4)

#> 
#> European Journal of Pharmacology   Journal of Medicinal Chemistry 
#>                                8                                7 
#>                Neuropharmacology               Psychopharmacology 
#>                                7                                7

The three sources were queried on 2 September 2026. The Scopus search for the function name in any indexed field illustrates why a bare string is a poor query: it returned 176 records, the largest group of them in pharmacology and medicinal chemistry, and not one of their titles mentions a mixed model. Scopus appears to match the string loosely against unrelated text, so those records are left out of the list below. The proxy query returned 1 record, which shows how rarely an abstract describes the treatment of convergence at that level of detail. The two full-text sources are the ones that come closest to answering the question, since they reach the methods section, where the function name is actually written. Their results are merged on the DOI and screened once, by dropping any record published before the function reached lme4, since a match in such an article is something other than a use of it.

# allFit reached lme4 in version 1.1-7, released in July 2014, so a record
# published before then cannot be a use of it, whatever matched the string.
allfit_year <- 2014L

used_allfit <- records |>
  filter(source %in% c('Europe PMC, full text', 'OpenAlex, full text')) |>
  mutate(key = ifelse(is.na(doi) | doi == '', paste(source, title), doi)) |>
  group_by(key) |>
  summarise(year = first(na.omit(year)), authors = first(authors),
            title = first(title), venue = first(na.omit(venue)),
            doi = first(doi), sources = paste(sort(unique(source)), collapse = '; '),
            .groups = 'drop') |>
  arrange(desc(year), authors)

too_early <- used_allfit |> filter(!is.na(year), year < allfit_year)
used_allfit <- used_allfit |> filter(is.na(year) | year >= allfit_year)
c(matched = nrow(used_allfit) + nrow(too_early),
  before_allfit_existed = nrow(too_early), kept = nrow(used_allfit))

#>               matched before_allfit_existed                  kept 
#>                    82                     6                    76

table(used_allfit$sources)

#> 
#>                      Europe PMC, full text Europe PMC, full text; OpenAlex, full text 
#>                                         13                                         16 
#>                        OpenAlex, full text 
#>                                         47

used_allfit |>
  filter(!is.na(year)) |>
  count(year) |>
  ggplot(aes(year, n)) +
  geom_col(fill = '#0072B2') +
  scale_x_continuous(breaks = scales::breaks_pretty(8)) +
  labs(x = NULL, y = 'Publications mentioning allFit') +
  theme_minimal(base_size = 13)

The table lists the publications kept by the full-text searches, most recent first, with the sources in which each was found. Authors are given as the sources returned them. The two sources overlap in a minority of records, which says as much about their coverage as about the literature.

short_authors <- function(a) {
  a <- ifelse(is.na(a), '', a)
  first <- sub('[;,].*$', '', a)
  ifelse(grepl('[;,]', a), paste(first, 'et al.'), first)
}
used_allfit |>
  transmute(Year = year,
            Authors = short_authors(authors),
            Title = title,
            Venue = venue,
            DOI = ifelse(is.na(doi) | doi == '', '',
                         sprintf('[%s](https://doi.org/%s)', doi, doi)),
            Sources = sources) |>
  knitr::kable()

The proxy query is shown for completeness. Its single hit is an erratum notice that carries the original article’s abstract, which is a fair indication of how rarely an abstract describes the treatment of convergence in this much detail. A wider proxy would be a reasonable place to look for further precedents, since studies that compared optimisers rarely name the function in a field that Scopus indexes.

records |>
  filter(source == 'Scopus, convergence proxy') |>
  transmute(Year = year, Authors = short_authors(authors), Title = title, Venue = venue,
            DOI = ifelse(is.na(doi) | doi == '', '',
                         sprintf('[%s](https://doi.org/%s)', doi, doi))) |>
  knitr::kable()

What the list misses

The count is a lower bound. Europe PMC holds full text mostly for open-access articles, and mostly in journals within its scope, so a study behind a paywall in a linguistics or psychology journal is usually invisible to it unless a preprint or an accepted manuscript was deposited. OpenAlex holds text for a subset of the literature that it does not fully document. Scopus cannot see the methods section at all, and a full-text match can be an article that uses the function in a context other than a convergence check, or that mentions it in a footnote. A study that compared optimisers with its own loop, or with the optimx package directly, is not found by any of the three, and neither is a study that reported the comparison only in supplementary material. Records published before the function existed have been dropped, but the table has not otherwise been screened by hand, so it lists the publications whose full text matches the name rather than the publications that are known to have used it. The list is therefore a starting point for the reader who needs a precedent, and it can be refreshed by dispatching the workflow again.