EqualifyEverything / benchmarks-ai-alt

Let a run stipulate how many accepted items it wants
250 accepted items is tens of hours of agent time. `./run.sh --target 100` sets a smaller goal, and the loop converges on it. The count targets scale from the 250-item baseline, so 100 items means 12 per category and 8 per sub-type rather than 100 items of whatever was easiest to find. Shares are ratios and do not scale. Floors stop a small corpus from degenerating into one item per sub-type, and a target under 25 is refused because seven sub-types at the floor already need 21: better a usage error than a loop grinding toward arithmetic that forbids the goal. The goal lives in corpus/target.txt, written by --target and read by validate.mjs, rather than only in a shell command. A goal that lived in one invocation would silently revert to 250 on the next round and read as ordinary slow progress. Committing it means every round, every bare validator run, and the history agree on what the run is working toward. A malformed target file is a usage error for the same reason. The validator prints the effective targets on every run and is authoritative where its arithmetic and directive 00's prose disagree. Directive 00 says plainly that a smaller corpus is a milestone, not v0.1, and supports weaker per-sub-type claims when published. Self-test: scaled targets stay reachable at every size, the target file is read strictly, and a stipulated goal persists to later rounds. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Blake Bertuccelli-Booth Blake Bertuccelli-Booth committed on Aug 27, 2026, 07:02 PM
Showing 5 changed files +234 additions -23 deletions
Browse files at this commit →