EqualifyEverything / equalify-iris-bench

fix(prepare): an `ok` record is a claim about a file, so check the file
Deleting the four 477 MB chunks left corpus.jsonl listing four paths nothing could open, because `ok` in prepared.jsonl is a claim made once and never revisited. The run stage would have discovered that one failed item at a time. The cache is gigabytes and gets pruned — deliberately now, when a splitter's output turns out to be unsendable, and eventually just for space — so a missing file is an ordinary state rather than a corrupt one. corpus.jsonl now excludes them, the summary says how many and from how many URLs, and --retry re-fetches and re-splits them even though the URL's own outcome was never a failure. Re-split with qpdf, the Texas act's chunks came out at 14 MB each rather than 477 MB — 34x smaller from the same 25 pages.
Blake Bertuccelli-Booth Blake Bertuccelli-Booth committed on Aug 24, 2026, 04:38 PM
Showing 2 changed files +49 additions -7 deletions
Browse files at this commit →