Recovering Published Numbers Did Not Ensure Major-Error Detection
Reproduction reuses original researchers’ data (recorded information) and code (computer analysis instructions) to recover reported numbers; fixing errors was not required. Brodeur and colleagues (published May 28, 2026) randomly assigned 288 researchers to 103 teams and teams to three work rules at seven 2024 events. Teams had seven hours. For human-only (no AI), unrestricted ChatGPT-assisted and ChatGPT-led teams, recovery counts were 31/33, 32/35 and 13/35 teams; average major errors found per team were about 1.70, 0.74 and 0.23, respectively. Major errors could affect claims. ChatGPT-led humans could not inspect papers/data/code. Models were GPT-4/GPT-4o, not 2026 agents. The small, selected known-error social-science sample limits generalization. No clear human-only/assisted reproduction difference was detected; equality was not established.
0
1
Tags
Scientific Oversight of AI-Assisted Research
1Cademy