Cross-language reproducibility • pilotr

library(pilotr)

The challenge

A toolkit that spans R and Python faces a subtle obstacle. The two ecosystems use different default random-number generators (R’s Mersenne-Twister with inversion, NumPy’s PCG64), so a direct port produces different data from the same seed. That would undermine the guarantee that a design authored once can be reproduced anywhere.

A shared generator

pilotr sets aside the native RNGs and provides a shared generator implemented identically in both languages. Uniform deviates come from L’Ecuyer’s (1988) combined linear congruential generator, whose arithmetic stays below $2^{53}$ and is therefore exact in IEEE-754 doubles (R) and Python integers alike. Normal deviates use Wichura’s (1988) Algorithm AS 241 inverse-CDF, the algorithm R’s qnorm() uses, so deviates agree to full double precision. The remaining draws (Cholesky-correlated random effects, inverse-CDF Poisson and ordinal draws, Marsaglia–Tsang gamma draws for the Beta family, a Fisher–Yates item sampler for partial crossing) are derived from those two through a documented, identical consumption order.

As a result, given the same specification and seed, the R and Python implementations produce bit-identical data sets. The specification, rather than the implementation, serves as the source of truth.

Determinism in R

The same specification and seed always give the same data.

spec <- build_spec(list(
  name = "demo", seed = 2024, design_kind = "between", n_subject = 200,
  factor_name = "group", lev1 = "a", lev2 = "b",
  intercept = 100, effect = 5, family = "gaussian", resp_name = "score", sigma = 10))

isTRUE(all.equal(simulate_design(spec), simulate_design(spec)))
#> [1] TRUE

Changing the seed changes the draws, while the structure and the ground-truth parameters stay fixed.

spec2 <- spec; spec2$seed <- spec$seed + 1L
identical(simulate_design(spec)$score, simulate_design(spec2)$score)
#> [1] FALSE

The same design in Python

The Python package reads the identical specification and produces the identical data. The code below is not evaluated here. It shows the Python you would run, and its output matches the R data to the last bit.

from pilotr import simulate, load_spec

# the same design.json the R package (and the app) use
data = simulate("design.json")

The repository includes a parity check that simulates the worked example designs in both languages and reports the maximum absolute difference per design.

design	rows	columns	max_abs_diff
between	64	3	0
crossed	1440	4	0
continuous	4000	7	4.885e-15
nested	4800	5	0
beta	200	3	0
partial	1440	4	0

The difference is exactly zero for every design whose responses are rounded, and about 5e-15 for the unrounded continuous design, where R’s CSV writer prints 15 significant digits. That residual is a number-formatting artefact of the comparison rather than a divergence between the generators, and it is what the check’s 1e-6 tolerance is there to absorb. There are no categorical mismatches anywhere. The uniform stream is exact by construction, since every intermediate product stays below $2^{53}$ , and the full pipeline has been observed bit-identical across the platforms tested. The parity check can be rerun on any new platform, so a design verified on a laptop can be confirmed on an HPC node before a large run.

Why this matters

Because the specification is the source of truth and the generator is shared, a design can be authored once (in the app, in R or in Python) and reproduced everywhere. It can be preregistered as a small JSON file that anyone can run, scaled from a laptop to a cluster without drift, and handed between collaborators who work in different languages.

This property turns an informal request to simulate some data into a reproducible, shareable methodological artefact.