Skip to content

The code-free app

Not every search starts in an editor. scopusflow ships a local NiceGUI app that drives the whole workflow through a browser tab, from describing a search to exporting the records, without writing any Python. It runs on your own machine, so your Scopus key never leaves it and requests come from your own network. The app doubles as a way into the library itself, because it mirrors every choice you make back as a runnable script. Anything you can do in the app you can later do from code, and the script in the panel shows you how.

Launch the app

The app lives behind an optional extra, so install it with the app group and start it with the console script:

pip install "scopusflow[app]"
scopusflow-gui

That opens a tab at http://127.0.0.1:8080. The script takes --host, --port and --no-browser, along with --version, so scopusflow-gui --port 9000 serves on another port and an unrecognised flag is refused outright. If you would rather start it from Python, call launch from the app module. It is not a top-level export, so reach it through the module:

import scopusflow.app

scopusflow.app.launch(host="127.0.0.1", port=8080, show=True)

By default, the app binds to 127.0.0.1, so it is reachable only from this machine. Another host, given as --host or as launch(host=...), makes the app reachable from other machines on that interface's network, and a key typed into it on one of them is sent across that network.

Try it in demo mode

Demo mode is on whenever the app opens in a tab, so you can walk the entire flow with no key at all. With it on, no request goes out at all: the harvest is replayed from the corpus bundled with the package, the live terminal still streams per-cell progress, the table and plots still render, and the Compare topics card still draws a figure. It is the quickest way to learn where each control lives before you spend any quota.

What the replay serves is real: 138 published articles on graphene supercapacitors, with their own titles, DOIs, journals and citation counts, taken from example_records. They are not a Scopus harvest, because retrieved records may not be redistributed, and Get started says where they do come from. The corpus covers 2015 to 2024, which is where the year slider opens, and a cell for a year outside that span comes back empty and says so in the live terminal, where padding it out with a neighbouring year's rows would double records up. With Partition by year off, the plan is a single cell with no year, and the replay serves it from the corpus's last year, 2024, so it returns that year's 14 records rather than all 138. Demo mode never contacts the network, so the records are the bundled ones whatever terms you type.

The window below is the app in demo mode just after a harvest, so everything in it was produced without a key. The search card sits on the left and the reproducible Python panel on the right, with the live terminal, the record count, the paginated table and the two figures filling the area beneath them.

The scopusflow app in demo mode. The search card on the left holds the API key field, the demo-mode switch, the search terms, the field selector, the Partition by year switch, the year range set to 2015 to 2024, the STANDARD and COMPLETE detail options and the Check plan, Fetch records and Cancel buttons. The Reproducible Python panel on the right shows the generated script, which calls pybliometrics.init(), fetches into a checkpoint directory of the plan's own, writes the PRISMA-S search record and compares the topics machine learning and deep learning, with a Download script button beneath it. Below them, the open Live terminal lists ten fetched cells and the 138 records retrieved. Then come a count of 138 records, a note that in demo mode those records came from the bundled example harvest and a paginated table of real article titles with their years, journals and citation counts. Two figures follow: records per year, which rises to a peak in 2019 and settles between thirteen and fifteen thereafter, and the most frequent journals, headed by ACS Applied Materials and Interfaces with eight records. Records, BibTeX, RIS and Search record download buttons sit beneath the figures, and the Compare topics card, with its terms filled in and not yet run, closes the page.

Demo mode hands back records with the same stable RECORD_COLUMNS schema a real harvest returns, so the table, the trend plot and the source plot behave exactly as they will against the live API. When you are ready for real results, paste your Scopus API key into the field at the top and switch Demo mode off. The key is held in the app's memory. It is never written to disk or to the generated script. If you have a pybliometrics configuration of your own, the app uses it without adding the pasted key to it. pybliometrics then files the responses in the cache that configuration names, where they stay after you close the tab. Without one, the app gives pybliometrics a key-less configuration and a cache in a temporary folder, which it removes when it stops. pybliometrics holds one configuration for the whole process, so the app assumes one active session at a time.

The left card is where you describe the search. You type your terms, choose which field to search in (title, abstract and keywords by default), and set a year range. The last entry in the field list, (none), sends the query as you typed it, which an expression carrying its own field tags needs. Detail chooses the Search API view. Under STANDARD, the default, each record names its first author alone. COMPLETE adds the full author list and an authkeywords column of author keywords, but it pages 25 records per request against STANDARD's 200, so the same harvest takes about eight times as many requests. The table on screen shows the same columns either way, and the Records (.csv) download carries the fuller author list and the extra column. In demo mode, the replayed records are the same under either choice, and only the script changes. Partitioning by year is recommended and switched on by default, because each year is then fetched, checkpointed and checked against the API's count separately, the same reasoning behind a partitioned SearchPlan in code. The year range applies only while you partition by year, so it appears and goes with that switch. The terms are trimmed of surrounding space before any request, so the search sent is the one the script panel shows.

Check plan sizes the search before you commit to it. In demo mode it counts how many bundled records the replay would return, which differs from year to year, since the corpus holds what the literature actually published. The answer stands for the plan on screen, so it goes as soon as the plan changes. With a real key it runs a single cheap count request, the same one scopus_count performs, and tells you how many records the query matches across how many year-cells:

import scopusflow as sf

# What "Check plan" runs with a real key (needs a configured Scopus key).
sf.scopus_count(
    "graphene supercapacitor",
    years=range(2015, 2025),
    field="TITLE-ABS-KEY",
)

That call contacts the Scopus API, so it only returns a number once pybliometrics holds a valid key. In the app, the key you paste is handed to pybliometrics in memory. From a script you call pybliometrics.init() once per session, before the first search, and it reads the key from your pybliometrics configuration.

Run the harvest with a live terminal

Fetch records starts the harvest. The retrieval runs off the event loop in a background worker, so the tab stays responsive while it works, and the Live terminal expansion streams a line per cell as each one starts. A progress bar tracks how far through the plan the run has reached, and Cancel stops the harvest once the cell in flight has finished, so a real run does not waste the cell it is already paying for. Cancel stops a harvest only. A plan check or a comparison, once started, runs to the end. A cancelled harvest ends with a warning such as "Stopped after 2 of 5 cells.", and its search record says how many cells ran, which years they covered and that the harvest is incomplete. Anything the harvest warns about appears in the live terminal as it happens, above all a cell that came back short of the total the API reports for it. The notice at the end then says that the harvest may be incomplete. Only one job runs at a time, since a harvest, a plan check and a comparison spend the same key, so each refuses while another is in flight.

Internally the app builds a SearchPlan from your choices and hands it to fetch_plan with a per-query cache directory and resume turned on. Fetching the same search again in the same tab, after Cancel or a failure, therefore resumes from the cells already checkpointed. Those checkpoints live in a temporary directory of the tab's own, removed when the tab closes, so closing one tab never touches the harvest running in another. A harvest that has to outlast the session, such as one stopped by the weekly quota, is the work of the generated script. The script keeps its checkpoints in a directory of the plan's own and resumes from them each time it is run. The equivalent in a script is:

import scopusflow as sf

# What "Fetch records" runs with a real key (needs a configured Scopus key).
plan = sf.SearchPlan(
    "graphene supercapacitor",
    years=range(2015, 2025),
    field="TITLE-ABS-KEY",
    partition="year",
)
records = sf.fetch_plan(plan, cache_dir="harvest-90624125", resume=True)
records.shape

When the run finishes, the results area shows the record count, a paginated table of titles, years, sources and citations, and two figures drawn by the library itself, a year trend from year_counts via plot_trend and the most frequent sources from top via plot_top. The trend covers the years the plan searched, and when they reach the year of the harvest, a note below it says that year was incomplete when the records were retrieved. A figure that cannot be drawn, as for records whose dates are all missing, is a short note in its place, and the table and the export buttons stay. Each figure is named for a screen reader by what it shows.

Read off the reproducible script

The right-hand card, labelled Reproducible Python, is the part that turns the app into a tutorial. It mirrors every choice you make, the query, the field, the year range, whether you partition by year, the chosen view, and the Compare topics options, and rewrites a runnable script as you go. Change a control on the left and the script updates immediately, so you can see exactly which argument each toggle sets.

The script the panel produces opens with a comment naming the scopusflow version that wrote it, and follows the same shape as the planning workflow, building a plan, fetching with a cache so a run resumes, inspecting the result, saving it, exporting it to BibTeX and writing the PRISMA-S search record:

import pybliometrics

import scopusflow as sf

# pybliometrics needs init() once per session. It reads your key from its
# configuration file, or creates the file and asks for the key on first use.
pybliometrics.init()

# Describe the search as an inspectable, reproducible plan.
plan = sf.SearchPlan(
    "graphene supercapacitor",
    years=range(2015, 2025),
    field="TITLE-ABS-KEY",
    partition="year",
)

# Retrieve, caching each cell so an interrupted run resumes. The
# checkpoints go to a directory of this plan's own.
records = sf.fetch_plan(plan, cache_dir="harvest-90624125", resume=True)

# Inspect the most frequent values and the records per year. These two
# lines show their result in an interactive session, and print nothing
# when the script is run with python.
sf.top(records, by="source")
sf.year_counts(records)

# Save the records.
records.to_csv("scopus-records.csv", index=False)

# Or export for a reference manager (Zotero, EndNote) or LaTeX.
with open("scopus-records.bib", "w", encoding="utf-8") as fh:
    fh.write(sf.to_bibtex(records))

# Write the search up for the methods section, to the PRISMA-S standard.
sf.scopus_search_report(records, file="scopus-search-record.md")

The script keeps its checkpoints in a directory of the plan's own, named harvest- and eight hexadecimal digits drawn from the query, the years, the field, the view and the partition. Running the same script again resumes from them, and a script for another search never overwrites them. The key is never written into the script. The script calls pybliometrics.init(), which reads the key from your own pybliometrics configuration, so a script you share carries the method but not your credentials. Download script (.py) saves exactly what you see, as scopusflow-script.py. If you rename it, avoid scopusflow.py. Python looks in a script's own folder first for anything it imports, so a script of that name would import itself where it expects the package.

Compare topics

The Compare topics card asks a different question from a harvest. It retrieves no records at all, and measures instead how a set of sub-topics co-occur with your search over time, as a share of it, with your search terms acting as the reference topic. You enter comma-separated comparison terms, optionally pick one to highlight, and toggle the stability band and whether each topic's label carries its record count. Each term needs one count request per year, and the reference topic one more, so two terms over six years take 18 requests. With Demo mode off, a note above the Compare topics button gives that number and advises fewer terms or years above 80. Demo mode sends no requests, so the note stays empty there.

The Compare topics card in demo mode. It holds a comma-separated comparison-terms field reading "machine learning, deep learning", a Highlight topic selector set to (none), switches for the stability band and for counts in the label and a Compare topics button. Beneath them is the resulting figure, showing the share of reference records held by each of the two topics rising from 2015 to 2024, deep learning from 13% to 19% and machine learning from 6% to 9%, each line labelled at its right-hand end with its record count and drawn inside a shaded stability band. A line under the figure notes that the demo counts are illustrative, and a Comparison (.csv) download button sits at the foot of the card.

In demo mode the comparison is the one part that is made up, since a comparison is a set of per-year counts that only the count endpoint can answer. The figure is drawn from invented numbers so you can see its shape offline, with its own progress streamed into the live terminal. With a real key the card calls compare_topics and draws the result with plot_comparison, the same figure the library produces:

import scopusflow as sf

# What "Compare topics" runs with a real key (needs a configured Scopus key).
cmp = sf.compare_topics(
    "graphene supercapacitor",
    ["machine learning", "deep learning"],
    years=range(2015, 2025),
    field="TITLE-ABS-KEY",
)
sf.plot_comparison(cmp, highlight="machine learning", interval=True)

A comparison always needs a year span, so with Partition by year switched off it covers the last six years, and the reproducible script names that same span. The highlight option only forwards a topic the plot can draw, so a term with no plottable share is quietly left unhighlighted, without raising. As with the harvest, the comparison contacts the Scopus API and needs a configured key to return real counts.

Export in one click

Every result the app shows comes with one-click export. The records table offers the frame as CSV, as BibTeX and RIS for a reference manager such as Zotero or EndNote, drawn from to_bibtex and to_ris, and as a PRISMA-S search record in Markdown, drawn from scopus_search_report. In demo mode, that record names no search expression, plan or retrieval time. It opens by saying that the records on screen were replayed from the bundled harvest and that it describes no search that was run. The harvest carries the phrase of the OpenAlex search that produced it, which is not a search anyone sent to Scopus. The Compare topics card offers its comparison frame as CSV. None of this needs the API again, because it works on results already in hand:

import scopusflow as sf

# The same export the download buttons perform, on records you already have.
with open("scopus-records.bib", "w", encoding="utf-8") as fh:
    fh.write(sf.to_bibtex(records))

with open("scopus-records.ris", "w", encoding="utf-8") as fh:
    fh.write(sf.to_ris(records))

sf.scopus_search_report(records, file="scopus-search-record.md")

Between the downloaded script and the exported records, a session in the app leaves you with both the data and the code that produced it, ready to drop into the rest of your workflow.

Beyond the app

Anything you can do in the app you can later do from code, but the reverse does not hold. The app covers planning, retrieval and the trend, top and comparison analyses, and the library goes further. scopus_intersections counts how a set of named concepts and their intersections overlap in the literature, and plot_scopus_intersections draws the result. scopus_abstract retrieves abstract text and richer metadata than the Search API returns, and corpus builds on it to give a harvest author keywords and structured references for tools further downstream. When a session in the app has taken you as far as it goes, the downloaded script is the place to carry on from.