Reproducibility | Atlas of Proteomic Technologies

Reproducibility

Every simulated or derived number this tool publishes (fairness win rates, co-recommendation rates, coverage sweeps, bootstrap confidence intervals) is reproducible from a downloadable engine specification and a self-contained script, without needing this repository or any part of the live app. This page collects that bundle for both recommendation engines, plus a separate descriptive pathway-coverage analysis that ships as source data rather than a spec-and-script pair (it feeds no engine, so there is nothing to regenerate), in one place. Platform scores themselves are not simulated; they are curated from published evidence and expert assessment, and their sourcing is documented dimension-by-dimension in Methods and per-study in the Evidence Base.

The data files the site is built from, with their schemas, are on the Data page.

Help Me Choose: recommendation engine fairness

Covers a full deterministic 1,259,712-scenario baseline grid (goals, sample types, study sizes, and toggle states weighted by real-world prevalence, not treated as equally likely; see Methods), a 4,374-scenario secondary check (plasma, biomarker discovery, large/extra-large study size, the single most prevalent real-world profile), and a rescoring of 65 anonymized real user sessions. The baseline grid and secondary check are complete enumerations, not samples, so they carry no seed and reproduce exactly; running regen_fairness.py against the engine specification below regenerates all five result files byte-for-byte, including the previous uniform-weighting methodology, kept as a documented, superseded reference. Full methodology, including how outright win and co-recommended rates are defined, why an earlier version's win-rate tallying was revised twice, and why the baseline moved from uniform to prevalence-weighted, is in Methods.

JSON

Engine specification

choose_engine_spec.json

Every scoring constant, goal adjustment, hard filter, prevalence weight, and the platform score matrix (engine v3.7).

Reproducible fairness script

regen_fairness.py

Regenerates all five result sets from the spec. Python 3, standard library only.

CSV

Anonymized real-world inputs

choose_real_world_data.csv

65 real Help Me Choose sessions: study design and priority weights only, no identifying fields.

CSV

Baseline grid outputs (prevalence-weighted)

choose_baseline_grid_results.csv

CSV

Baseline grid outputs (superseded, uniform)

choose_baseline_grid_uniform_legacy.csv

Same two statistics under the previous uniform-weighting methodology, n=314,928. Kept for audit only; do not use for reporting.

CSV

Secondary-check outputs

choose_secondary_check_results.csv

Same two statistics for the plasma / discovery / large-xlarge slice, n=4,374.

CSV

Real-world rescoring outputs

choose_real_world_results.csv

Same two statistics, computed from the 63 scored anonymized real sessions.

CSV

Tie-group-size distribution

choose_tie_distribution.csv

Scenario-level: share of each dataset resolving to an outright win vs. a 2- through 6-way tie.

Help Me Combine: pairing engine fairness

Covers a deterministic priority sweep (924 scenarios: every combination of up to three priorities, over all four coverage denominators), plus bootstrap confidence intervals on coverage. The priority sweep reproduces exactly, no seed involved; the bootstrap draws reproduce under the seed recorded in the script. A random-weight sweep was previously part of this bundle and has been withdrawn: two of the seven axis weights it drew uniformly are zero-weighted by default and reachable only when a user selects them, so it does not represent a weighting the priority picker can actually produce, and it is not reported in the paper. Full methodology is in Methods, under Platform Combination Engine.

JSON

Engine specification

combine_engine_spec.json

Axes, default weights, and scoring formulas (engine v3.7).

Reproducible sweep script

regen_fairness.py

Regenerates the priority sweep and the coverage CIs from the spec. Seeded, Python 3, requires numpy.

CSV

Priority-sweep outputs

fairness_priority_sweep.csv

Win, top-3, regret, and rank per pair and platform across 924 priority scenarios.

CSV

Coverage confidence intervals

coverage_bootstrap_ci.csv

2,000-resample bootstrap 95% CI per pair, for the drug-target and FDA-biomarker denominators.

Pathway coverage analysis (Figure 6C, 6D, Supplementary Table S11)

Source data behind Figure 6's platform-specialisation and pair cost-coverage-frontier panels (6C, 6D) and Supplementary Table S11, together with the full pathway-by-platform and pathway-by-pair tables those panels are drawn from: 1,285 testable pathways checked for enrichment across 6 platforms, and per-pathway coverage for all 15 platform pairs across 1,393 pathways. Pathway coverage throughout this bundle is computed against the canonical proteome, that is, each pathway's full Reactome membership; this matches the denominator used by the live pathway-coverage feature in Help Me Choose and Help Me Combine, so recomputing coverage from these files should use the same canonical reference set. These are descriptive analysis outputs, not an input to either recommendation engine: pathway coverage does not enter the Help Me Choose or Help Me Combine scoring formulas, and nothing in this bundle is consumed by either engine's reproducibility script. Full methodology for the enrichment and specialisation analysis is described in a companion preprint.

CSV

Pathway enrichment by platform

reactome_enrichment_by_platform.csv

Hypergeometric enrichment of each platform's covered proteins against 1,285 testable Reactome pathways: observed vs. expected count, fold enrichment, and raw/FDR-adjusted p-values (n=6,713 platform-pathway rows, 6 platforms).

CSV

Per-pathway pair coverage

pathway_pair_coverage_full.csv

Best single platform and best platform pair for every one of 1,393 pathways, each with its coverage percentage and the gain from pairing over the best single platform alone.

JSON

Pathway accession-to-name map

reactome_pathway_names.json

Maps each Reactome stable identifier (e.g. R-HSA-1059683) to its full pathway name, 1,242 entries. The enrichment and pair-coverage files below carry their own name columns too, but the enrichment file's is blank for 817 of its 1,285 pathways; this map resolves 293 of those, leaving 524 (40.8%) identifiable by accession only.

JSON

Figure 6C source data

panelC_specialisation.json

The 18 platform-pathway pairings behind Figure 6C: each platform's most specialised pathways, with coverage percentage, effective-coverage score, fold enrichment, and pathway size.

JSON

Figure 6D source data

panelD_frontier.json

Per-pair druggable-assay count, median and 90th-percentile pathway coverage, and cost-coverage frontier membership, all 15 platform pairs.

CSV

Supplementary Table S11 (protein-class coverage)

supp_table_S11_class_coverage.csv

Best single-platform and best-pair coverage by protein class, the six-platform union, and an affinity-vs-mass-spectrometry coverage-skew label, 11 classes.

CSV

Druggable-target family partition

druggable_target_family_partition.csv

How the druggable target set splits by which platform family reaches it: affinity-only, mass-spectrometry-only, both, or neither (292 / 152 / 713 / 293).

Verifying a result yourself

Download the engine specification and the script for the tool you want to check into the same folder, then run the script (python3 regen_fairness.py). It writes its own output files and prints a summary table; compare either against the downloadable result files above. If you find a discrepancy, email chris@ignitionscientific.com with the script output and we will investigate.