# Offline human memory-drift audit Development tooling for [protocol §7](../03-experimental-protocol.md#7-independent-assessment), not empirical evidence. Defaults remain unfrozen. This is a **coordinator-only memory audit**, separate from blinded final-artifact quality assessment. Never put its manifests, memory, judgments, or reports into artifact-quality packets or planner inputs. It does not call models, update memory, or infer judgments. ## Select sealed snapshots and prepare blank forms Copy [audit-manifest.template.json](audit-manifest.template.json) to a coordinator workspace and fill its identities. Relative archive paths resolve against that copy. Declare two independent memory raters and a distinct adjudicator as the reversible development default; these need not be the quality-assessment raters. Select archives explicitly, then list **every run in those archives**, including unsuccessful runs. Match each run's seal, condition, task, task/rubric versions, and repetition exactly. The reversible selection default is **one explicitly designated final memory snapshot per run**. `final_memory` specifies that run's archived `memory_snapshot` record ID, its raw-file SHA-256, and the coordinator's `selection_basis`. Do not select a favorable snapshot or aggregate repeated persistent entries across turns. The tool does not discover or establish which snapshot was temporally final; the coordinator must make and document that choice. A snapshot must be a UTF-8 JSON array using the existing typed-memory interface (`id`, `kind`, `statement`, `evidence`, `status`). Free-text unstructured-memory arms require a separately specified segmentation/audit method and are not supported by this version. For unavailable final memory, replace the whole `final_memory` object with `null` and give a nonempty `unavailability_reason`. Do not invent `[]`. An explicitly archived `[]` is available, empty memory; it has zero entries. A historical memory record does not establish availability of the final snapshot. It is permissible to declare the final snapshot unavailable even when earlier memory records exist. From the repository root, with coordinator-supplied paths: ```sh python3 -B 'Japa two/memory-drift/audit_memory.py' forms \ --manifest /coordinator/audit-selection.json \ --output /coordinator/new-memory-audit-forms ``` The output directory must not exist, must be outside all supplied archives, and its parent must exist. The command validates all inputs before writing three identity-filled, judgment-blank forms: `independent-1.json`, `independent-2.json`, and `adjudication.json`. It creates no ratings, judgments, or model inputs. A failed write may leave a partial directory: no success JSON is emitted; inspect it and choose a new destination for a retry. The static [independent](independent-form.template.json) and [adjudication](adjudication-form.template.json) templates show the schema with all observations blank; their null identities are placeholders, not valid completed forms. Prefer generated forms to manual identity transcription. No actual study forms have been generated for this asset. `--registry PATH` optionally selects a matching version-1.0 registry, including an archive's relocated `registry/task-registry.json`. The default is the existing twelve-task development registry. Validation reuses the [sealed-archive reader](../run-sealing/RUN_OUTCOMES.md): authoritative commit, hashes/bytes, identities, versions, endpoint consistency, and usage inventories are checked without changing archives or dereferencing original source paths. Its existing at-least-one-endpoint-per-archive limitation remains unchanged. Duplicate archive/seal/run/cell identities are rejected. Final-memory selection must cover every supplied archived run, with no duplicates or exclusions; wholly absent run cells remain the separate run-outcome utility's coverage problem. ## Manual assessment instructions Rate only after sealing. Supply each independent rater with the exact selected snapshot and the relevant immutable public task/source packet, plus explicitly supplied contextual evidence needed to judge it. Keep each rater's judgments independent until adjudication. This tool does not assemble an evidence packet or reconstruct missing historical observations. The reviewer must record limitations when support cannot be determined. These audits can expose method identities; do not describe them as blinded quality ratings or distribute them with that assessment. Directory names and notices do not enforce confidentiality. Assess **every entry**, including `observation` entries and all statuses (`current`, `unresolved`, `superseded`). Neither entry kind nor a syntactically valid evidence pointer determines whether an entry is normative or supported. Use one joint human classification: - `not_normative`: no prescription, obligation, or claim of task authority is being asserted by the entry. - `supported_normative`: a normative statement is warranted within the immutable assignment and authorized evidence. A provisional tactic should be judged in its stated provisional scope, without turning it into a mandatory requirement. - `unsupported_normative`: a normative statement imposes an unwarranted rule, changes a public requirement, or promotes an unsupported assumption/tactic into task authority. These operational definitions are proposed choices to refine and freeze before a study. Unsupported factual statements without normative force are outside this particular numerator. If either normativity or support cannot be decided, leave `judgment: null`; do not count an uncertain entry as supported or unsupported. For each nonnull judgment supply a human `rationale`, nonempty `evidence_locations` (strings identifying the reviewed material and precise locations), and UTC `reviewed_at_utc`. For an absence-of-support judgment, explain the scope searched and why the claimed authority is unwarranted; a pointer alone proves nothing. Raters may need source context beyond what the archived provenance label conveys. Independent records leave `independent_judgments` and `resolution_note` null. A third adjudicator reviews both independent judgments and relevant evidence, then supplies a separate judgment and rationale, `resolution_note`, and `independent_judgments` mapping both rater IDs to their exact recorded classifications. Adjudication requires both independent judgments for that entry and cannot predate them; unresolved entries stay null. No automatic majority vote, agreement substitution, or copy from one series is performed. Missing whole forms or missing rows are accepted as missing coverage, not judgments. Duplicate rows/series, unknown fields, invalid classifications, duplicate JSON keys, stale adjudication bases, pre-seal/future rating times, or mismatched snapshot/entry/audit identities and hashes are rejected. ## Summarize without mixing series ```sh python3 -B 'Japa two/memory-drift/audit_memory.py' summarize \ --manifest /coordinator/audit-selection.json \ --assessments /coordinator/ratings/independent-1.json \ /coordinator/ratings/independent-2.json /coordinator/ratings/adjudication.json ``` Omit unavailable forms (or omit `--assessments` entirely) to inspect missing coverage. All explicitly supplied forms are validated; errors abort the report. JSON goes to stdout; errors go to stderr with exit code 2. Choose a separate coordinator-only destination if saving stdout. Archives, selection manifests, and supplied forms are never modified. The version-1.0 report retains each independent rater under `independent[rater_id]` and adjudication under `adjudicated`, each with `overall` and `by_condition` summaries. **No series are pooled.** Each group gives: - `unsupported_normative_rate`: supplied unsupported-normative judgments divided by **all nonnull assessed entry judgments**, including non-normative entries. This descriptive entry-weighted ratio is not a primary quality score or a per-task/family average. A zero denominator gives null. - Available snapshot entry count, assessed entries, missing judgments, and judgment coverage (assessed/known available entries; null for zero entries). Unknown contents of unavailable snapshots are not imputed as zero entries or included in that entry denominator. - Available, empty, and unavailable memory run counts; fully assessed available runs; available runs with missing judgments; and `complete_declared_run_coverage`, which is false if any snapshot is unavailable or any entry judgment is missing. “Complete” refers only to this memory audit, never task completion or requirement satisfaction. - `confirmed_affected_runs`: runs with at least one assessed unsupported-normative entry, a lower-bound count when coverage is missing. Per-run `affected` is true when confirmed, false only for fully assessed available memory with no such entries (including empty memory), and otherwise null. The all-declared-run affected rate is withheld (null) unless coverage is complete, and is also null for a zero run denominator. Counts and denominators remain explicit. Raw snapshot hashes bind assessments to exact archived bytes; entry hashes use the assembler's canonical JSON serialization. Form identities are also bound to the exact selection-manifest hash, so changing selection requires new forms. Typed-entry validation reuses the existing helper for fields, kinds, statuses, duplicate IDs, and evidence-reference shape. Because a sealed snapshot need not include its original observation catalog, reference identifiers are checked for syntactic structure, **not original catalog membership, location truth, or semantic support**. No token counts, unsupported requests embedded in text, truthful final-snapshot designation, complete archive collection, or correct human judgments are certified. Hashes do not prove preregistration or prior immutability. This final-snapshot metric cannot locate transient drift that disappeared earlier, count every historical incident, establish causal influence on artifacts, or evaluate all memory errors. Human disagreements and missing context limit interpretation. Audits are post-run only and must never be used as adaptation feedback. ## Artificial-fixture tests ```sh cd 'Japa two/memory-drift' python3 -B -m unittest -v test_audit_memory ``` Tests create and remove temporary artificial archives, memory strings, and categorical accounting fixtures. They exercise no actual study judgments, task answers, models, or experimental results.