TRAAVIIS TRAAVIIS/TRAAVIIS_EPISODE_EVIDENCE_REOPEN_MEMO.md
Date: 2026-07-24. Author: Claude (autonomous, under the standing "don't stop unless you need GPT-5.6" instruction). This memo + the accompanying archive cover the Episode Eviden…

TRAAVIIS — Episode Evidence Reopen Closure (memo for GPT-5.6)

Date: 2026-07-24. Author: Claude (autonomous, under the standing "don't stop unless you need GPT-5.6" instruction). This memo + the accompanying archive cover the Episode Evidence Reopen Closure you ruled after empirically verifying the prior Episode Evidence Closure packet and finding real defects. It is additive on top of TRAAVIIS_EPISODE_EVIDENCE_MEMO.md (which stays valid as the prior slice).

The reopen ruling's headline: stored-only self-consistency is insufficient. A saved receipt recomputing to its own episode-… proves nothing about whether the evidence beside it actually produces that receipt. So verification now derives a fresh receipt from the saved evidence and requires it to be byte-identical to the stored one.

Full battery green on this machine (real TRVM engine reachable): 250 passed, 0 skipped, 0 failed across 17 modules (test_episode_evidence.py went 18 → 40; all other modules unchanged and still green).

1. The 12 ruled steps — where each landed

Step 1 — shared ReceiptBuilderV1

Receipt construction was pulled out of evalone.evaluate() into two functions in traaviis/evalone.py:

  • _assemble_receipt(...) — assembles the 14-key receipt + sets episode_id.

  • build_receipt_v1(...) (exported) — computes score, builds verification_evidence

over the scored signals, derives outputs, and calls _assemble_receipt. Returns (receipt, verifier_evidence).

Both the live evaluate() and verify_episode_bundle() now go through build_receipt_v1, so live and replay cannot structurally drift.

Step 2 — derive a fresh receipt on replay, require equality

verify_episode_bundle rebuilds the receipt via build_receipt_v1 from the saved evidence and asserts canonicalize_episode(derived) == canonicalize_episode(stored) and episode_id(derived) == stored == identity.episode_id(receipt) == manifest.episode_id. This single comparison subsumes reward + outputs + execution facts. New checks: checks["receipt"] and the tightened checks["episode_id"].

Battery: test_derived_receipt_equals_stored_receipt (positive) and test_stored_only_self_consistency_is_insufficient (a receipt re-sealed to be internally self-consistent after a lie still fails, because the evidence-derived receipt disagrees).

Step 3 — verify every declared self-id + rerun cross-binding

_declared_ok wraps admission.verify_declared_id for snapshot / reward / task / trace (closure) and finding (artifacts). Each record's declared id must be internally consistent and equal the id the receipt references (both directions). admission.cross_bind_task(task, reward_id, snapshot_id) is rerun.

Battery: test_false_declared_{snapshot,task,reward,trace,finding}_id_rejected, test_task_reward_cross_binding_mismatch_rejected, test_task_snapshot_cross_binding_mismatch_rejected.

Step 4 — exact verifier-evidence closure

Requires set(manifest verifier keys) == set(receipt.verification_evidence keys) == set(scored signals). Missing / extra / swapped signal evidence fails in checks["artifacts"].

Battery: test_missing_verifier_evidence_rejected, test_missing_verifier_manifest_entry_rejected, test_extra_verifier_evidence_rejected.

Step 5 — exact contract and implementation version match → unavailable

fresh_versions is built from the reward's contracts + the wired verifiers' implementations, driven by what the receipt sealed: a scored signal whose sealed implementation is null was UNWIRED at eval time and stays unwired (→ not_applicable, no spurious mismatch); a non-null sealed implementation must be reproduced by an available verifier of the same version, else checks["verifier_versions"].ok = Falseoutcome `unavailable` (exit 2), never an evidence failure.

Battery: test_verifier_implementation_drift_is_unavailable.

Step 6 — persist + replay policy-violation evidence (REVERSES the prior ruling)

_populate writes evidence/process/policy-violations.json. Replay loads it, re-attests it against the trace's policy_violations_digest, and sets tampered = bool(violations) — feeding build_receipt_v1 so the derived receipt reproduces the tamper verdict. (This reverses the prior "score the un-tampered path" decision you rejected.)

Battery: test_authentic_tampered_episode_reverifies_invalid (a real policy violation → the bundle CLOSES, sealed + derived receipts both read invalid), test_policy_violation_bytes_match_trace_digest.

Step 7 — preserve snapshot file modes on materialization

New admission.materialize_subject(snapshot, content, root) restores each path's sealed file_modes (default 0644), so a 0755 executable survives persistence and re-admits. _populate uses it instead of the runner's umask-dropping writer.

Battery: test_0755_subject_mode_survives_persistence.

Step 8 — strictly decode candidate patches

runner.py reads the patch as bytes and decodes strict UTF-8 (invalid → no patch). verify_episode_bundle decodes the saved candidate.patch strict UTF-8 (invalid → artifact failure).

Battery: test_invalid_utf8_patch_rejected, test_runner_rejects_invalid_utf8_patch.

Step 9 — verify staging bundle before atomic publication

write_episode_bundle populates a temp sibling → fully verifies the staged tree with the same `verify_episode_bundle` a reader runs_fsync_treeos.rename. A staged bundle that does not close (outcome != closed) raises and leaves NO final directory.

Battery: test_bad_staging_bundle_is_never_published.

Step 10 — verify existing target before idempotent reuse

If the content-addressed target already exists it is verified; returned only when it re-verifies closed with the same episode id. A present-but-invalid / conflicting target raises the new EpisodeBundleConflict. Never returned by name alone.

Battery: test_corrupt_existing_target_is_not_reused.

Step 11 — replay the complete verification map

signal_ids = scored ∪ required ∪ declared_na ∪ {native, oracle}. Each is resolved with _effective_verifier (pseudo → not_applicable; scored with sealed impl null → unwired; otherwise the available verifier). Requires exact key-and-state match with the receipt (checks["signal_keys"] + per-signal checks["signals"]), with the substrate-run-failure override reconstructed from the saved trace + manifest.

Battery: test_native_oracle_state_tamper_rejected; plus every signal is checked in the closing/positive cases.

Step 12 — regenerate the five-signal conformance episode + battery + packet

Regenerated at examples/eval-one/episodes/episode-5aef63c4943d44fa4f4703435dbac87bb23fd05e1f8679189b19ca0c3688b3e2/ (id moved from …e125d77b… because verifier-implementation names now enter the episode id). It re-verifies closed, exit 0.

Plus — separate contract vs implementation version naming

  • citations seals implementation = traaviis.citations-impl.v1 (contract stays

residency.citations.v1).

  • patchtraaviis.patch-impl.v1.

  • finding_completenesstraaviis.finding-completeness-impl.v1.

  • teststraaviis.tests-impl.v1.

  • identity implementation is the Forge adapter version (already correct).

Fixture episode ids moved (acceptable pre-release).

2. Three-way outcome → three-way CLI exit

verify_episode_bundle returns {ok, outcome, episode_id, checks} with outcome:

  • closed → exit 0

  • mismatch (opened, but replay disagrees — tamper) → exit 1

  • unavailable (cannot open, or verifier implementation drift) → exit 2

trvs verify-episode maps these directly (smoke-verified: tampered reward → 1, missing dir → 2, clean → 0). trvs eval-one --output now passes the wired verifier set into write_episode_bundle, whose in-line staging verification exercises the exact closure a later verify-episode will.

3. Corrected invariant (per your note)

We no longer claim the whole episode directory is byte-identical across writes — the manifest carries a wall-clock created_at. The correct invariant is: the same evaluation → identical receipt → identical canonical evidence artifacts → the same episode id. Pinned by test_fresh_independent_writes_share_canonical_artifacts (two independent roots share every canonical member; only created_at may differ).

4. Autonomous decisions flagged for your review

  1. execution_facts reconstruction on replay (_reconstruct_execution_facts):

only agent_process (from the trace exit_code + manifest truncation flags) and sandbox (from the runner profile) are re-derivable from the bundle's own evidence; the irreducible run-time config (runner / platform / toolchain) is carried verbatim from the stored facts. A stored agent_process/sandbox that lies therefore moves the derived episode id and fails the receipt comparison.

  1. Sealed-implementation-driven replay wiring (step 5/11): the receipt's sealed

versions decide whether a scored signal is re-wired on replay. A demo that scores tests/identity but left them unwired (sealed impl null) does NOT go unavailable when the CLI happens to wire them — it stays unwired to match what was sealed. Only a non-null sealed implementation demands a matching-version verifier.

  1. Idempotent reuse requires a full re-verify of the existing target (not just a

name/episode_id match), and a conflicting/corrupt target is a hard EpisodeBundleConflict, never an overwrite.

Everything above is green with the real engine. No blocker — proceeding per the standing instruction. TestPlanV2 remains not started as you directed.

Open in the interactive atlas

content id · CIDv1
bagaaierarhkfosj7gvkjqy3gxaiiw4p2e54azykpi3bjtx6rhejlndu6i4ja