# Offline first-observed cost to quality threshold Untested development tooling for [protocol §7](../03-experimental-protocol.md#7-independent-assessment); outputs are **not measured study findings**, and defaults remain unfrozen. This is retrospective coordinator-only analysis, never adaptation feedback. It validates existing human assessments and actual billing records; it does not create judgments, execute models, estimate prices, or change the prescribed final endpoint. ## Inputs and request boundaries Copy [manifest.template.json](manifest.template.json) to a coordinator workspace. It contains placeholders, not actual study inputs. Supply: - A raw-file-hash-bound [progress analysis manifest](../progress-per-turn/README.md). Its unchanged validator checks sealed snapshots, chronology, human forms, public-requirement mappings, rater aliases, artifact hashes and post-seal assessment times. Its task/condition/repetition panel and turn horizon define expected coverage here too. No missing run, turn, or rating is filled. - An explicit numeric `quality_threshold` in `[0,1]`, inclusive. Booleans, strings, nulls, nonfinite and out-of-range values are rejected. The default comparison is primary normalized requirement satisfaction **≥ threshold**. Decimal input is preserved; validated count/12 scores are compared as exact rationals, avoiding float-rounding threshold artifacts. The threshold is a proposed choice to prespecify before substantive analysis, not selected from results. - Explicit version-1.0 [resource ledgers](../resource-accounting/README.md), each with `ledger_id`, path and raw SHA-256. Paths resolve against this manifest's parent. Ledgers may reconcile billing after execution; their hashes do not claim that the reconciled bills were originally sealed. All ledger runs must match supplied archive identities. One ledger per run is the reversible default; combine provider/chunk records explicitly beforehand without duplicate requests. - One `run_accounting` entry for **every supplied progress run**, matching its archive/seal/run/condition/task/version/repetition identities. Select its ledger, or use `ledger_id: null` with a nonempty unavailability explanation. A supplied ledger containing that run contradicts a declaration that its ledger is unavailable. `adaptation_request_order` explicitly lists every sealed adaptation attempt ID exactly once, in submission order, across all roles and outcomes. Include failed attempts, cancelled calls, retries, rejected comparisons/proposals and discarded candidates; do not select only retained worker calls. Request IDs must be unique across supplied ledgers/archives. Ledger records are validated by the unchanged accountant; adaptation records must match the sealed attempt's run, phase, role and retry parent. Retry parents must precede their retries. Supplied start times must respect the declared order. Ledger array order is not assumed chronological. For each supplied delivered snapshot, a boundary identifies its turn, exact artifact record/hash, and `cumulative_request_count = k`: the first **k requests** in that run's declared adaptation order. Counts are nonnegative integers, bounded by order length, and nondecreasing across chronologically listed boundaries. Every request in that prefix is charged regardless of disposition/status. A boundary cannot omit an explicitly artifact-linked worker attempt. A boundary with unknown position uses a null count and explanatory reason; omitted boundaries are also missing accounting, never inferred from a neighboring turn. The reversible timing default is a **closed request prefix**. With known clocks, included requests must have started and ended by snapshot delivery, and excluded requests cannot have started earlier. This rejects in-flight or inconsistent allocations; partial/concurrent-request cost allocation is not implemented. Equal-clock ties use the explicit prefix. Unknown clocks rely on the coordinator's declared order and linkage, not fabricated times. Hashes and attempt membership cannot establish true causal attribution or exhaustive request logging. An explicitly complete adaptation ledger must contain all sealed adaptation attempts. An incomplete ledger may omit records, subject to the existing retry-link contract; omitted calls retain unknown billing. As a conservative default, `phase_log_complete.adaptation: false` withholds **all prefix totals**, even when some prefixes have known subtotals. Do not assume its omissions occur only after a chosen hit. Actual billed costs can be known while token usage is unknown; no token price is inferred. Independent `final_assessment` requests remain separate and never enter adaptation prefixes. Post-seal assessment charges can have new IDs, but cannot reuse/mislabel a sealed adaptation request. Their per-run resource summaries are reported separately. ## Run locally ```sh python3 -B 'Japa two/cost-to-quality/analyze_cost_threshold.py' \ --manifest /coordinator/threshold-selection.json \ --output /coordinator/new-threshold-analysis.json ``` `--registry PATH` optionally selects the compatible registry used by progress analysis; the default is the existing twelve-task development registry. The output parent must exist; the destination must be new and outside source archives. Existing paths, including broken symlinks, are refused. The report is staged, verified and atomically published without overwrite; failures emit no success record. Stdout contains only the output path/hash and development labels. Errors go to stderr with exit code 2. Keep outputs outside rater packets; coordinator-only labels do not enforce filesystem permissions. Inputs and completed utilities are unchanged. ## Interpretation and coverage For each independent rater and separately for adjudication, the default event is the **first assessed delivered snapshot meeting the threshold**. Its cost is cumulative **actual billed adaptation cost through its explicit request boundary**. Negative later quality changes are retained in the snapshot record. The tool does not assume monotonic quality, interpolate between snapshots, choose a cheapest/best historical snapshot, or substitute a later costed hit when the first observed hit's accounting is missing. Each run has two explicit classifications: - `quality_outcome`: observed attainment when a qualifying assessed snapshot exists; fully observed non-attainment when every expected turn has an eligible score and none qualifies; otherwise indeterminate. - `threshold_analysis_outcome`: observed attainment only when that witness also has complete prefix billing; fully observed non-attainment only with complete quality and accounting; otherwise indeterminate. Thus a known quality witness can coexist with indeterminate cost. It is never reclassified as failure or zero cost. `first_observed_attainment` identifies turn, artifact, score and actual currency-separated costs, or remains null for no observed witness. Missing earlier assessments are flagged: the first observed hit need not be the earliest qualifying declared snapshot. Missing costs remain null, with known subtotals/unknown-record diagnostics kept separately. Fully observed non-attainers receive **no attainment cost**, neither zero nor an invented infinite value; full-run adaptation spending is reported separately as observed expenditure. Currency amounts stay in separate exact-decimal strings. Mixed currencies produce a vector, never a converted or combined scalar. Missing amounts, missing currencies and incomplete logs withhold cost totals. A genuinely complete empty request prefix has a known empty cost vector and a zero-request flag, but no invented currency. No pooled cost mean is produced: per-run costs and condition-level outcome/coverage summaries avoid disguising censoring, currency incompatibility or partial cohorts as a complete cost estimate. Condition summaries retain the full expected-run denominator from progress analysis, observed/complete/incomplete counts, and the three outcome counts. A trajectory is complete only when all declared quality turns, snapshot boundaries and full adaptation billing are complete, including work after the first hit. Cohort attainment rates are withheld unless the relevant declared coverage is complete. A known early hit/cost can still be reported when later data are missing, while the cohort stays incomplete. Quality-only rates are labeled separately and can remain known when billing is missing. Zero denominators yield null; unsuccessful and absent cells are never silently dropped. All final-endpoint identities/hashes remain those of the sealed archives. This retrospective event does not replace final-budget quality scoring or implement a stopping policy. First observed attainment does not prove persistent quality, an exact crossing time, causal improvement, actual review correctness or a scientifically justified threshold. Actual billing and complete-history declarations remain coordinator/provider evidence; structural validation cannot certify their truth. No actual study inputs, assessments or findings were generated for these assets. ## Temporary artificial-fixture tests ```sh cd 'Japa two/cost-to-quality' python3 -B -m unittest -v test_analyze_cost_threshold ``` Fixtures contain temporary artificial text, numeric ratings and accounting cases, not task answers, measured study costs or experimental results.