pilotr: Simulate experimental and behavioural data from a portable design specification

Package Web application Open Materials
Bernabeu, P. (2026) — Version 0.3.1 on the Comprehensive R Archive Network (CRAN) and Version 0.3.0 on PyPI [Computer software] software
Code DOI R documentation Python documentation Browser app CRAN PyPI Blog post

Abstract

Twin R and Python packages for simulating experimental and behavioural data from a portable design specification. pilotr supports crossed mixed-effects designs, power and precision analyses, and reproducible generation from the same specification in either language.

Overview

pilotr (Bernabeu, 2026a, 2026b) makes a planned study executable before any data are collected. A portable specification describes the units, predictors, response family, fixed effects and random-effects structure, and the same specification can then be simulated and analysed in R or Python. The main output is a design diagnosis that goes beyond a single power percentage to cover detection, precision, Type S (sign) and Type M (magnitude) errors, and model warnings.

Detection, Precision and Exaggeration

A simulation-based design analysis should establish how often the target effect is detected, how precisely it is estimated and how much the statistically significant estimates exaggerate it. Figure 1 reports detection across participant counts, Figure 2 precision over a wider range of counts, and Figure 3 the exaggeration that survives a significance filter.

Illustrative power curve for a priming effect across sample sizes, with an 80% reference line
Detection Probability Across Participant Counts. Each design uses 12 replicates. All simulated designs contain 24 items.
Two-panel precision plot showing the probability of excluding a negligible effect and mean confidence-interval width as sample size increases
Precision and the Probability That an Interval Excludes a Prespecified Negligible-Effect Region. Each design uses 12 replicates. All simulated designs contain 24 items.
Type M error declining towards 1 as power increases across effect-size and sample-size simulations
Magnitude Exaggeration Among Statistically Significant Estimates. Each design uses 12 replicates.

Reproducible Planning

Treat the specification, simulation seed, package version, number of replicates and fitted model as part of the planning record. The companion blog post keeps its examples quick by running each analysis with a small number of replicates. A real design decision requires many more replicates and a sensitivity analysis over plausible assumptions. The R documentation and Python documentation cover the full workflow, and the browser app, which needs no installation, builds and simulates a design and estimates power for a simple two-group comparison.

References

Bernabeu, P. (2026a). pilotr: Simulate experimental and behavioural data from a portable design specification (Version 0.3.1) [Computer software]. Comprehensive R Archive Network (CRAN). https://doi.org/10.32614/CRAN.package.pilotr

Bernabeu, P. (2026b). pilotr: Simulate experimental and behavioural data from a portable design specification (Version 0.3.0) [Computer software]. PyPI. https://pypi.org/project/pilotr/0.3.0/

Documentation

R documentation Python documentation Browser app
software R Python simulation power analysis linear mixed-effects models

Comments are provided by Disqus and are not loaded automatically. Loading them connects your browser to Disqus, which may use cookies and process data under its privacy policy. See this site's privacy notice for details.