# Offline primary-contrast analysis Version 1.0. **Development analysis tooling, not experimental evidence.** Software tests use artificial numeric scoring-report fixtures, not task answers or study observations. No study estimate or empirical report is supplied. This adds [analyze_primary_contrast.py](analyze_primary_contrast.py) beside the existing scorer without changing scoring behavior, registry, blank forms, guides, or prompts. The [protocol's statistics section](../../../03-experimental-protocol.md) defines the primary contrast as C's repetition-mean requirement-satisfaction score minus B1's for each task, with a 95% task-clustered, family-stratified bootstrap interval. The coordinator must map the explicit supplied IDs to those intended conditions. The utility does not infer methods from names, choose the most favorable comparison/series, perform secondary multiplicity correction, or establish superiority/noninferiority. ## Usage Python 3.9+ and the standard library are sufficient. From this folder, analyze a completed version `1.0` report emitted by [score_forms.py](score_forms.py): ```sh python3 -B analyze_primary_contrast.py --report /path/to/scoring-report.json \ --treatment-id C --control-id B1 --series adjudicated \ --replicates 10000 --seed 0 ``` For one independent-rater series, use `--series independent --rater-id RATER_ID`. Both condition IDs and `--series` are mandatory. Exactly one series is used: one named independent rater **or** adjudication. A rater ID is required for independent analysis and forbidden for adjudicated analysis. There is no pooling, averaging, or fallback across raters or adjudication. Prespecify the analysis series before inspecting outcomes. `--registry` optionally selects the associated task registry; the default is the existing `task-registry.json` beside the utility. Only its version/task/family metadata is used, not answer guides or rating criteria. Shared strict JSON reading and validation exceptions are imported from the existing scorer; no scoring function is invoked. The analysis reads no rating forms and recalculates no satisfaction judgments. It uses already reported task means, cross-checking their consistency against already scored repetition rows and aggregate summaries. Output is coordinator-only JSON on standard output, with source file hashes, selected IDs/series, paired task/repetition metadata, family differences, mean difference, interval, and bootstrap configuration. Exit 2 produces an error on standard error and **no estimate** if validation fails; exit 0 produces a complete analysis. Inputs are read-only; if redirecting output, choose a new file, never an input path. Keep both the report and analysis outside rater/worker/planner packets. A seal ID is a declaration, not proof of prior sealing or independence. ## Pairing, weighting, and coverage For each task `i`, `d_i = reported_mean[C,i] - reported_mean[B1,i]`. The point estimate averages these task differences within each family, then averages the three family means equally. Repetitions are equally weighted inside each task, and tasks equally weighted inside each family. The current registry specifies twelve tasks, four per family. Repetitions are not additional independent task draws. For each bootstrap replicate, sample four whole task clusters with replacement **separately within each family**. A sampled cluster carries both conditions and all their paired repetition IDs; do not resample arms, artifacts, raters, or repetitions independently. Its fixed `d_i` is mathematically sufficient for this linear statistic, so the code resamples validated task differences rather than repeatedly copying the underlying rows. Averaging within strata and then equally across strata preserves family weighting. Both selected conditions must report complete coverage in the chosen series, with no missing assignment/record/incomplete lists. All twelve registered tasks and the same nonempty repetition-ID set must occur in both conditions and every task. Every selected score must be finite, in [0,1], eligible, and have no missing criteria. Unknown tasks, duplicate artifact/record IDs or paired cells, nulls/booleans, mismatched counts, inconsistent summaries, and incompatible report/registry versions are rejected. Missing coverage is never imputed, dropped, or repaired using another score series. Other conditions may remain incomplete and are not part of this contrast. Report version `1.0` embeds repetition rows and coverage summaries but not the original manifest or registry fingerprint. These are sufficient for cross-checking paired rows against the reported counts and registered task membership; the analysis cannot reconstruct hidden declarations or prove that a hand-edited report reflects a frozen design. Use the exact completed scorer output and its matching registry. No registry label or report version is silently upgraded. The protocol proposes at least three whole-run repetitions; the utility permits smaller development fixtures but does not certify them as a registered study. ## Reversible development bootstrap defaults Defaults are **10,000 replicates**, local PRNG **seed 0**, and a percentile interval at 2.5% and 97.5%, linearly interpolated at zero-based rank `q × (B−1)` in sorted bootstrap statistics. Replicate count is configurable (integer ≥2); seed is configurable (integer). These are explicitly **unfrozen, reversible development choices**, not a power calculation or a paper-mandated interval variant. Freeze them before study analysis. Very small counts are accepted for software checks and are unsuitable for stable tail estimation. Canonical sorted family/task/repetition order makes results invariant to report row order for the same inputs/configuration. The bootstrap uses a local seeded `random.Random` instance, not global RNG state. Record the Python version along with the output for reproducibility. Intervals describe uncertainty under this task-sampling scheme; a degenerate interval or a few development tasks does not establish reliable generalization. This tool supplies no p-value or post-hoc stopping/selection procedure. ## Software checks ```sh python3 -B -m unittest -v test_primary_contrast.py test_score_forms.py ``` The focused numeric fixtures check comparator/task pairing, retention of all repetitions, family/task/repetition weighting, seeded reproducibility and order invariance, percentile interpolation, series isolation, missing coverage, versions, duplicates, and read-only/error behavior. They are temporary software inputs, not fabricated study results. No model, experiment, API spending, production change, or submission is involved.