Question: In a concise analytical response, explain why repeatedly checking the same reviewed subset can lead to overfitting and how keeping a separate untouched subset helps detect it.

Sample answer: If developers keep studying one reviewed subset, their tuning choices can gradually fit that subset too closely. A separate untouched subset provides a second reference point that is not used for manual inspection. If performance looks better on the reviewed subset than on the untouched subset, that gap suggests the team may be overfitting to the cases they keep examining.

Key points:

• Repeated review can cause the examined subset to be fit too closely
• A separate subset is kept untouched by manual inspection
• The untouched subset gives an independent reference for tuning
• A gap between the two subsets can reveal overfitting from analysis

Rubric: A strong response explains that frequent manual inspection can bias tuning toward the reviewed cases, describes the role of the untouched subset as an independent check, and connects a performance gap to overfitting caused by the analysis process.