# Offline request-resource accounting Version 1.0. **Development tooling, not measured study costs.** Tests use artificial accounting fixtures, not API receipts or experimental observations. No study-cost ledger or measured-cost report is supplied. Interfaces/defaults are unfrozen. This tool reads completed ledger copies; it does not launch models, execute experiments, estimate prices, grade artifacts, enforce a live budget, or change existing scoring/contrast tools. ## Usage and scope Python 3.9+ and its standard library are sufficient. From this folder: ```sh python3 -B account_resources.py --ledger /path/to/completed-ledger.json python3 -B -m unittest -v test_account_resources.py ``` Copy [ledger.template.json](ledger.template.json) and fill/duplicate its run/request records. It is intentionally invalid while blank. The utility reads only the supplied JSON and emits a coordinator-only JSON summary on stdout. It never writes inputs. Redirect output only to a new path. Exit 2 reports a JSON validation error on stderr and emits no totals; exit 0 can include unknown totals, so inspect completeness and missing counts. No subscription price, list price, exchange rate, or assumed free call is used. The contract follows [protocol §6](../03-experimental-protocol.md) and the completed [B0–B2](../baseline-prompts/README.md), [B3](../baseline-prompts/b3-spo/README.md), and [B4](../baseline-prompts/b4-feedback-descent/README.md) call/accounting conventions. The output summarizes **each run**, **each condition across runs**, and **all runs**, each split into `adaptation` and `final_assessment`, with totals and separate role summaries. Final-assessment expense is study expense outside the adaptation budget. There is no grand total that mixes these phases or currencies. ## Version 1.0 ledger contract The root has exactly `ledger_version` (`"1.0"`), `ledger_id` (nonempty string), `runs` (array), and `requests` (array). IDs are coordinator metadata, kept outside rater/worker/planner packets. Field names are exact; duplicate JSON keys and unexpected/missing fields are rejected. Each run has a unique `run_id`, explicit nonempty `condition_id`, `task_id`, and `repetition_id`, plus `phase_log_complete` with boolean `adaptation` and `final_assessment` flags. A condition/task/repetition combination identifies one run; retries remain in that run. At least one run must be declared. Completeness is an explicit operator declaration, not inferred from success, output quality, or silence. Mark a phase false if collection is incomplete or not yet reconciled. A complete phase with no requests is a known empty log; an incomplete empty phase is not evidence of zero expenditure. Each request has: | Fields | Contract | |---|---| | `request_id`, `run_id` | Unique nonempty attempt ID; reference to a declared run | | `phase` | `adaptation` or `final_assessment`, explicitly classified by use of the feedback | | `role` | `worker`, `planner`, `optimizer`, `critic`, `comparator`, `selector`, `scorer`, `artifact_rendering`, `tool`, or `other` | | `status` | `succeeded`, `failed`, `cancelled`, or `unknown` | | `disposition` | `retained`, `rejected`, `discarded`, `not_applicable`, or `unknown` | | `retry_of` | null for an original attempt, otherwise the request ID of an earlier logical attempt in the same run/phase/role; record order is immaterial | | `provider`, `model_snapshot`, `pricing_date` | Reported metadata or null; date, if supplied, is a real `YYYY-MM-DD` calendar date, never an instruction to retrieve pricing | | `usage` | The complete counter/relationship/duration object below | | `billed_cost` | `{ "amount": number or null, "currency": uppercase three-letter code or null }`; actual recorded request charge only | One row records **one actual attempt/event**, not one successful deliverable. Use stable instrumented attempt IDs to prevent duplicate ingestion; fresh IDs cannot make a duplicated receipt a genuinely new attempt, and the tool cannot detect that deception without raw logs. Missing/cyclic retry links and cross-run/phase/role links are rejected. Status and disposition never filter expenditure. Distinct retries count separately even if their responses were rejected or incomplete. Repeated comparisons after B4 disagreement are new comparison calls; use `retry_of` only for actual retries, not algorithmic repeated judging. Unknown role-specific usage must remain null. Explicit zero is permitted only when actually known; do not record non-model operations as free simply because no model usage is reported. For B2, log critique and revision separately. For B3, log the optimizer, every worker candidate (including rejected ones), all four comparator requests, and any optional model selector. For B4, the proposal **is** the candidate worker execution: log it once, not as two calls; log every swapped-order comparison and any optional model acceptance serializer separately. Capture failed/transport attempts and retries as well. Deterministic local selection is not a model request; if its rendering/tool work is separately instrumented, use that operation's own role record and avoid charging the same cost again elsewhere. A comparator used for adaptation is never reclassified as independent final assessment to make the baseline cheaper. A scorer role alone does not determine phase. ## Token overlap and missing usage `usage` has exactly `input_tokens`, `cached_input_tokens`, `output_tokens`, `reasoning_tokens`, `cached_input_relationship`, `reasoning_relationship`, `wall_time_seconds`, and `tool_runtime_seconds`. Token counters are nonnegative **integers or null**, rejecting booleans, floats, strings, negatives, and non-finite values. Durations and billed amounts are finite nonnegative JSON numbers or null. Currency codes are checked for format, not against an online currency catalogue. Credits/refunds or shared invoice charges require a separately declared allocation/reconciliation policy; do not duplicate a batch charge into every request. Relationships are explicitly `included`, `additional`, or `unknown`: - `cached_input_relationship` describes whether the cached counter is already included in the reported input counter or is additional to it. - `reasoning_relationship` does the same for reasoning versus reported output. - Inclusive parent plus subset: normalized total for that side is the **parent only**. A known subset cannot exceed a known inclusive parent. A missing subset does not erase a known inclusive parent. - Additional counters: normalized side total is parent plus additional counter, only when both are known. - Unknown relationship: normalized side total is null; do not guess the provider's counting convention. `normalized_tokens.nonoverlapping_total` adds normalized input and output only when both are known. Raw reported counters are also summarized separately, with relationship counts; **never add all four raw counters** to obtain a total. Raw parent-counter sums may mix provider conventions and are descriptive, not a normalized consumption total. Record documented semantics at ingestion; the tool does not infer them from a model name. Unknown cache/reasoning usage is not zero, and a token count is not a billed-cost estimate. Each numeric summary has `known_records`, `unknown_records`, `known_subtotal`, and `total`. Total is null if any contributing record is unknown **or** any relevant run's phase log is incomplete. An entirely unknown nonempty set has a null known subtotal. A partial subtotal is explicitly partial; it cannot certify budget compliance. Request counts count recorded attempts exactly, with a separate `request_count_complete` flag. Durations are decimal strings and sum request durations; summed wall time is not elapsed run time when requests overlap. Image usage units and provider-specific modalities are outside this text-study ledger version and require an explicit versioned extension rather than token conversion guesses. ## Actual billing and budget interpretation Billing is summed exactly as decimal strings **separately for each reported currency**. `single_currency_billed_total` exists only for a complete, fully known, single-currency group. Multiple currencies retain separate totals and no converted/combined total. Amounts with unknown currency are preserved individually, never summed; they also prevent claiming a complete total for any currency, since their allocation is unknown. Known-currency records with unknown amounts retain that currency's known subtotal and a null total. An empty group has no inferred currency or fabricated zero-dollar total. An incomplete run phase makes the corresponding condition/role totals incomplete even if its recorded requests have known usage. There are no mean costs per task or equal-budget claims inferred from an unbalanced collection. Compare adaptation totals only after checking declared run coverage, logging completeness, compatible currencies, settings, model snapshots, and actual billed-cost availability. The software cannot establish that an operator logged every request, allocated shared costs correctly, or honored a prespecified cap. Freeze these collection policies before study execution; keeping a ledger is not evidence of an equal-budget experiment. Tests use temporary numeric fixtures to check overlapping counters, unknown usage, currency separation, exact decimal addition, rejected/failed/retry costs, run/condition/role grouping, final-assessment separation, malformed input, and input immutability. No completed study ledger, task answer, empirical cost estimate, model call, API spending, production change, or submission is produced.