# Offline human repeated-work assessment Untested development tooling for [protocol §7](../03-experimental-protocol.md#7-independent-assessment); **all defaults remain unfrozen**. Repeated work means **redoing already completed work without a task-supported reason**, coded by humans after sealing. No text-similarity rule, judgment generation, model call, progress score or adaptation feedback is implemented. The utility reuses sealed-archive, packet, exact-span, JSON-identity, binary-rating and timestamp helpers without changing existing utilities. ## Declare the chronology Copy [selection-manifest.template.json](selection-manifest.template.json) into a coordinator workspace. Fill two independent raters and a distinct adjudicator as the reversible default. Explicitly name archives and selected run/seal/condition/task/repetition identities and versions. Selection may be a prespecified subset; unselected archived run IDs are reported separately. Unsuccessful runs remain eligible; no best artifact or successful-run filter is inferred. The default unit is **one adjacent delivered-artifact transition**, not a request, retry, discarded candidate, or arbitrary pair of snapshots. For each run: - Declare `trajectory_status` as `reviewed`, `unfinished`, or `unavailable`, with a nonempty private `trajectory_note` explaining scope/completeness. Reviewed is a coordinator declaration, not proof of complete history. - For reviewed/unfinished trajectories, declare integer `declared_delivery_count = N ≥ 0`. Delivery positions are 1…N in chronological order. List supplied `deliveries` in increasing `sequence`, naming unique sealed artifact record IDs and hashes. Identical bytes under distinct captured record IDs are allowed. Omit unavailable delivery slots rather than renumbering later ones; gaps remain visible. Never relabel discarded candidates as delivered outputs. - Explicitly list each supplied transition with a globally unique source `transition_id`, `earlier_sequence = i`, `later_sequence = i+1`, in increasing order. Known expected coverage is all `max(N−1,0)` adjacent slots, even when some declarations or artifacts are missing. A 1→3 pair cannot bridge a missing second artifact. - List submitted worker instructions for each transition in submission order. Each descriptor names its sealed `submitted_prompt` record/hash and `attempt_id`; it must belong to the same run and the named adaptation-worker attempt. Retried/failed worker submissions can be listed; they do not create extra units unless another artifact was actually delivered. Record IDs can be reused across distinct attempts, but duplicate `(record_id, attempt_id)` events are rejected. - Set `instruction_status: "complete"` only for an explicitly complete, nonempty list, with null issue note. `unfinished` allows a partial/empty list and requires an explanatory `instruction_note`; `unavailable` requires an empty list and explanation. Initial instructions before the first delivery are outside these between-delivery lists; the immutable task packet supplies the fixed assignment context. Chronological order is explicitly caller-declared, a reversible default because archive inventories do not encode delivered-turn order. UTC `delivered_at_utc` and `submitted_at_utc` can be null when unknown; do not invent times. Supplied clocks must be strictly increasing in their declared order, precede sealing/run end, and put instructions between all relevant known delivery bounds. Known attempt start/end times also constrain submissions. Unknown clocks remain unknown and do not invalidate otherwise complete declared ordinal coverage; the tool cannot verify actual ordering from them. For an unavailable trajectory, use `declared_delivery_count: null` and empty delivery/transition lists. Its transition count is unknown, not zero. An unfinished trajectory can supply provisional units, but never yields complete-run rates. A reviewed run with zero or one delivery has no adjacent unit and a null transition rate. This says nothing about repeated internal computation within a single call. ## Build blank assessment packets ```sh python3 -B 'Japa two/repeated-work/assess_repeated_work.py' build \ --manifest /coordinator/transition-selection.json \ --rater-output /coordinator/new-repeated-work-raters \ --coordinator-output /coordinator/new-private-repeated-work-key \ --seed 0 ``` Both outputs must be new, disjoint, have existing parents and lie outside source archives. Existing paths, including broken symlinks, are refused. [Shared sealing validation](../run-sealing/RUN_OUTCOMES.md) checks committed archives, payload/reference hashes, identities, run cells and versions; existing archive limitations remain unchanged. Artifacts need not be designated final endpoints: they must be explicitly selected sealed members of that run. `--registry PATH` optionally selects a compatible registry, including an archive's relocated registry; otherwise the existing twelve-task registry is used. Relative archive paths resolve against the original manifest parent. Each opaque rater directory contains unchanged artifacts and explicitly selected submitted-worker instructions, the matching unchanged public packet, an index, instructions, and blank coding forms. Known adjacent slots with missing inputs remain in the index/form as `input_complete: false`; they cannot receive a nonnull code. No candidate/history discovery or text stripping occurs. Chronology within a trajectory is retained; the unfrozen seed shuffles whole trajectory presentation blocks. Fresh UUID aliases are not seeded. Condition/run/repetition mappings, source record/attempt IDs, actual clocks, private notes, memory, resource records, traces and quality scores stay outside rater files and filenames. Relative delivery positions are necessary assessment context. Coordinator output preserves the exact selection and opaque-ID map; `repeated-key.json` is written last. Failures may leave partial directories but no success output; inspect and choose new destinations rather than overwrite. Directory separation does not enforce confidentiality. Artifact **and instruction** text can intrinsically reveal methods; it is preserved, not heuristically redacted. This is condition-mapping separation, not a guarantee of successful blinding. ## Human coding and evidence Generated forms follow the blank [independent](independent-form.template.json) and [adjudication](adjudication-form.template.json) schemas. Templates contain placeholders, not completed study records. Code integer `repeated_work: 1` only when evidence supports already completed work, an actual redo in the next artifact, and no task-supported reason. Code `0` for a justified/nonrepeated transition; unresolved cases stay null. Necessary rechecking, repairs, changed assumptions or task-required restatement may justify revisiting work. Mere textual retention, equal hashes, instructions requesting repetition, and stylistic rewriting do not by themselves prove repeated work. Do not infer unseen computation. Each nonnull code requires `reason`, `reviewed_scope` and UTC completion after sealing, followed by `locked_at_utc`. Positive codes also require both: - `completed_work_evidence`: exact earlier artifact locations from the same trajectory, no later than the before position. - `redone_work_evidence`: exact locations in the designated after artifact only. Each artifact reference has `artifact_id`, `sha256`, `start`, `end`, `text`. Offsets are zero-based Unicode code-point `[start,end)` into exact UTF-8-decoded bytes, including CR/LF/BOM; no normalization occurs. Optional `task_reason_evidence` has `channel`, `document_id`, `sha256`, `start`, `end`, `text`: channel is `public_packet` with this task ID, or `instruction` with an instruction alias supplied for **this transition**. Explain absence of a reason and what was reviewed when no precise positive justification span exists. These references validate location, not completion, redo, or task authority. Raters can inspect supplied earlier context within the trajectory; later artifacts cannot establish earlier completion and are rejected as completed-work evidence. Looking ahead can still bias a human and is not prevented by filesystem layout. If omitted initial context, unavailable work history or ambiguous instructions prevent a defensible judgment, leave it null and report the limitation. Submitted instructions show requests, not proof of executed actions or automatic authority to change the immutable task. Rate independently and lock forms before sharing. The third human records a separate code, evidence, resolution note and `independent_judgments` mapping both opaque independent aliases to their exact original binary codes, after both locks. No agreement copying or majority vote is performed. Adjudicating disagreements alone leaves the remaining adjudicated units missing. Never return any coding to adaptation. ## Summaries and limits ```sh python3 -B 'Japa two/repeated-work/assess_repeated_work.py' summarize \ --key /coordinator/new-private-repeated-work-key/repeated-key.json \ --forms /coordinator/returned/independent-a.json \ /coordinator/returned/independent-b.json /coordinator/returned/adjudication.json ``` Forms may be omitted, including the whole `--forms` option, to inspect coverage. Original selection files can be removed: the byte-exact coordinator copy and original path base are retained. Source archives and a compatible registry must remain accessible. Summary revalidates them and reconstructs expected identities without writing packets or changing inputs. Duplicate records/series, reversed/nonadjacent chronology, invalid evidence, extra metadata fields, wrong hashes/identities/versions, invalid binaries, stale adjudication and unsealed inputs are rejected. JSON goes to stdout; errors go to stderr with exit code 2 and no report. Save stdout only to a new coordinator destination outside inputs; shell redirection is not managed. Reports separate every independent rater from adjudication, overall and by condition. They provide repeated-work counts and assessed-transition denominators; known expected and available units; missing input, missing judgment and combined missing-transition coverage; and confirmed affected-run counts. Missing input counts and missing judgments are disjoint. Unavailable trajectories have null per-run expected counts and separate unavailable-run counts; known totals exclude their unknown units rather than treating them as zero. Reviewed no-transition runs remain explicit. Rates are descriptive transition-weighted ratios, not primary task/family scores or a progress index. Complete transition/affected-run rates are withheld unless every selected trajectory is reviewed, all declared deliveries/transitions/instructions are supplied, and all eligible units are judged. Zero transition denominators return null. Affected counts are lower bounds under missing coverage; a run's absence of observed repetition is null until complete. The affected-run rate's explicit denominator is selected runs, including reviewed no-transition runs; its scoped zero is not evidence against hidden internal rework. Completeness means declared coverage, not proof that the coordinator enumerated every delivery. `independent_rater_agreement` is separate from adjudication: exact binary agreement over jointly nonnull transition ratings, explicit denominator, missing joint judgments and incomplete-input/trajectory context. Zero joint denominator gives null, not perfect agreement. Overlapping transitions share artifacts, and transitions/ratings within runs and task repetitions are dependent; these descriptive counts are not independent-sample estimates and no confidence intervals are supplied. This metric codes human evidence of observable redo from delivered artifacts and submitted instructions. It cannot establish hidden execution effort, distinguish recomputation from copying solely by text, certify completeness/semantic purity of mislabeled records, prove chronology or task-supported reasoning, quantify all internal repetition, or establish superiority. No actual study packets, judgments or findings were generated for these assets. ## Temporary artificial-fixture tests ```sh cd 'Japa two/repeated-work' python3 -B -m unittest -v test_assess_repeated_work ``` Fixtures are temporary artificial text/metadata and binary accounting cases, not task answers, model executions or empirical results.