A Second AI Doubted 1 in 5 Test Enquiries. We Deleted Those.
Correction, 24 October 2026. Section 2 gave exp_040’s counts from an earlier filter run, and the original title (“Two AIs Disagreed on 1 in 5 Test Enquiries”) and summary called the deleted enquiries disagreements. Both now say doubted. The trained set. exp_040 trained on 2,267 items. Sampled six times each, the untrained model got 1,296 right every time and 349 wrong every time, which left 622 items with any spread. The 2,104, 1,281, 270 and 553 we gave are from filter run 2, before 163 abstention items were added (exp_040 BUILD.md, results/training/filter_manifest.json). The base still solved most items: 57 %. What the 85 were. In exp_043 all 85 deleted enquiries were ones the second reader flagged as doubtful. The two readers disagreed on the request in 6 of them, and on no label at all in 70 (exp_043 data/raw/, data/chk/, and HYPOTHESIS.md: “85 dropped on checker doubt”). In exp_044 all 98 were dropped on doubt. Records: exp_040 filter manifest, exp_043 pre-registration. ...