Comparison

BioMysteryBench’s Human-Difficult Label Is Panel-Specific

A benchmark tests performance on specified tasks. Anthropic’s BioMysteryBench report (first published April 29, 2026; last modified October 9, 2026) retains historical results for 99 biological-data questions. Up to five field experts tried each from scratch: 76 were solved by at least one person; 23 remained unsolved by that panel, labeled human-difficult after initial checks. On those 23, Anthropic reports Claude Mythos Preview’s roughly 30% average per-attempt solve rate, using five attempts per question. This tests answer-finding, not understanding a supplied AI answer; equal time and resources were not established. The page is mutable, not a frozen April snapshot. This rate does not describe the later corrected 90-task version.

0

1

Updated 2026-11-09

Contributors are:

Who are from:

Tags

Scientific Oversight of AI-Assisted Research

1Cademy