Learn Before
Coding Completion and Immediate New-Skill Learning Diverged
Shen and Tamkin’s posted manuscript (preprint v2, February 1, 2026) reports chance assignment for programmers learning Trio, an unfamiliar coding toolkit. Its registration describes grouping by experience and knowledge before assignment. The AI condition added GPT-4o assistance that could inspect code and supply solutions, with encouragement to use it; both groups had instructions and web search. Among 52 analyzed people, 26 per group, all assisted participants finished the second task; four controls did not. The assisted group scored lower on an immediate quiz without AI; average time saving was not established. This concerns acquisition, not existing-skill decay. Interaction styles were chosen, not assigned. Registration-timing statements conflict, and the total originally assigned to each group remains unclear.
0
1
References
Tags
Scientific Oversight of AI-Assisted Research
1Cademy