# GB1 retrospective assay benchmark — protocol v1 Fixed before the first benchmark execution on November 7, 2026 UTC. - Source: [Wu et al. (2016), Supplementary file 1](https://doi.org/10.7554/eLife.16965.024), original `elife-16965-supp1-v4.xlsx`. SHA-256 `ac9a16295cd24110ad3ef4bcc57741f023ff9f3f5c3d9f5fdc0a94b996dca9b7`. - License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), as linked by the [primary article](https://elifesciences.org/articles/16965). Credit: Nicholas C Wu, Lei Dai, C Anders Olson, James O Lloyd-Smith and Ren Sun. Changes: read measured table, reconstruct parent-aligned sequences, sample train/test sets, fit regression. No endorsement implied. - Outcome: the original, untransformed selection enrichment relative to wild type. This assay reflects folding and IgG-Fc binding; it is not a binding constant, isolated stability measure, organismal fitness, or therapeutic outcome. - Include the 149,361 measured rows, including zero values. Never read supplementary file 2 (imputed outcomes). No imputation or target-dependent sampling. - Model: `additive-ridge-v1`, alpha 10.0, amino-acid substitution indicators relative to parent, unpenalized intercept, maximum 200 cyclic coordinate-descent sweeps, tolerance `1e-7 * max(1, training outcome range)`. No label transformation or feature scaling. No tuning after test inspection. - Primary regime: random within-scaffold interpolation, 384 measured training variants and 1,000 distinct test variants, sampled without replacement with Python `random.Random` seeds 20261107, 20261108, 20261109. - Stress regime: 384 training variants at Hamming distance 0–2 from WT and 1,000 test variants at distance 3–4, same seeds. This intentionally evaluates extrapolation the deployed shortlist gate excludes. - Split before fitting; feature vocabulary/support learned from training data only. No sequence occurs in both train and test within a run; data across seeds can overlap and is not independent replication. No claim of family-held-out performance: all rows are GB1. - Comparators: training mean, nearest training sequence (tie-average), current DeltaFold composition heuristic (rank only), expected random shortlist mean. - Report MAE, RMSE, Spearman correlation, measured outcome mean in the top 24 predicted variants, better-than-WT count in that shortlist, ranges over seeds, worst errors, excluded-domain counts, and convergence. A constant prediction has undefined Spearman, reported as null. - Failure criterion: no improvement over simple baselines or poor extrapolation must be disclosed. Offline performance cannot establish customer demand, clinical benefit, or prospective laboratory advantage. Reproduction (Python 3.11+; no third-party packages): ```text python scripts/benchmark_variants.py --data /path/to/elife-16965-supp1-v4.xlsx --output benchmarks/gb1 ``` Original data download: https://cdn.elifesciences.org/articles/16965/elife-16965-supp1-v4.xlsx The source workbook is deliberately not shipped in the application. `results.json` records split IDs/checksums/model coefficients; `predictions.csv` contains the retained test predictions. Re-running replaces these derived benchmark artifacts, not project data. ## Numerical amendment After the first execution, a software test identified a convergence-tolerance issue for assay values much smaller than 1 in the chosen units. The stopping rule was corrected to `1e-7 * (max(y)-min(y))` for nonconstant outcomes, and exact rather than thresholded equality now detects constant labels. This changes numerical scale handling, not model selection, alpha, sampling, or features. The benchmark was rerun with the correction; both execution results are equivalent at reported precision.