EqualifyEverything / benchmarks-ai-alt

An item recorded by URL alone stops being scoreable the day its page changes, and the change is silent: nobody can tell afterwards whether the gold standard was right about the image the model was actually given. So the corpus now archives the bytes. tools/fetch-images.mjs copies each image into corpus/images/, names the copy after its item, and writes image_file and image_sha256 into the record. Nothing else writes that directory. The checks are split by what they can see. validate.mjs checks the shape of the reference and adds a corpus-level criterion, so an accepted item with an image URL must have a copy. fetch-images.mjs --verify checks the bytes on disk against the recorded hash. The loop runs both every round, and apply-verdicts.mjs refuses to promote an item whose image was never copied. A URL that has stopped resolving is an ordinary outcome: the item keeps its URL and waits. A copy whose bytes changed under us stops the round, because every judgment about that item was made against the old ones. The directives, the schema and the READMEs say all of this, and AGENTS.md names the directory in its one exception to the plain text rule. Self-tests: 12 validator, 10 apply-verdicts, 10 second-pass, 9 fetch-images, 22 loop. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>