Example

SPECS: Target Recall Is Not All-Objection Precision

Biswas and colleagues’ author-posted draft (arXiv v1, 2026-04-15) reports SPECS: 783 deliberate target edits to versions of 120 earlier research reports. An AI judged whether generated reviews—written critiques—caught each planted target. A system using several review steps detected 63.86% of targets versus 42.91% with a single review instruction. Recall means the share of targets detected; precision means the share of all raised objections—claims of a problem—that are justified. Two humans checked 35 edits: both judged 22 significant errors and 9 too minor, and disagreed on 4. That small audit concerns target labels, not all raised objections; neither result supplies precision for unedited reports.

0

1

Updated 2026-11-09

Contributors are:

Who are from:

Tags

Scientific Oversight of AI-Assisted Research

1Cademy