EqualifyEverything / benchmarks-ai-alt

Make the second gold standard pass a real, separate turn
Directive 00 requires two independent gold standard passes per accepted item, and independent means the second was authored without reading the first. One agent turn cannot do that: round 01 produced pass-a and pass-b from the same context, so agreement was guaranteed and the requirement was satisfied only on paper. - directives/02-second-pass.md: a new role, run between seeking and review, on an input file that carries page context and nothing else. - tools/second-pass.mjs: --extract writes that file from an allowlist of context fields; --apply merges the passes back, refusing any pass for an item the round never extracted, and reports which items now disagree. - Disagreement is a legal candidate state and an illegal accepted one. The validator only enforces resolution on acceptance, and apply-verdicts refuses to promote an unadjudicated disagreement. The next round's seeking agent adjudicates; the reviewer treats such items as revise, never accept. - run.sh runs three turns per round, with --prompt second-pass and --merge-passes N for a hand-run round, and stops the loop when the second pass turn writes a log but no passes. - The reviewer now checks independence it can actually check: a pass-b that appears in no round's second pass file is coded NO-SECOND-PASS. - Round 01's 12 pass-b entries were withdrawn, its 3 promotions reverted, and its 5 adjudications moved into notes. Recorded in corpus/corrections.md. Directives renumbered: adversarial review is 03, the loop is 04. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Blake Bertuccelli-Booth Blake Bertuccelli-Booth committed on Aug 27, 2026, 07:32 PM
Showing 14 changed files +1181 additions -96 deletions
Browse files at this commit →