EqualifyEverything / benchmarks-ai-alt

Review round 3: judge a round by its artefacts, not its exit code
Round three looked at the failure modes a stub agent cannot show, and found three ways the loop could go quiet or lose evidence. - An agent that exits zero without writing anything is now caught. In print mode a denied tool permission looks exactly like a successful turn, so the loop checked the one signal that does not distinguish them. Each round now requires its artefacts, and says what to do when they are missing. - A reviewer that writes no review records while candidates are pending stops the round. Conversely, no candidates and no records is now a legitimate quiet round rather than a spurious failure. - Round numbering counts seek logs as well as reviews, so a round that dies between seeking and review does not have its log overwritten by the next attempt. rounds/ is the audit trail. - A review record that is both `accept` and `blocking` is now a schema error. It would have promoted the item and held it against the 95 percent criterion at once, with no later round able to clear it, because accepted items are not reviewed again. - Fix the help text, which printed a stray `set -u` line. Self-tests: 24 cases, all passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Blake Bertuccelli-Booth Blake Bertuccelli-Booth committed on Aug 27, 2026, 05:46 PM
Showing 5 changed files +122 additions -8 deletions
Browse files at this commit →