Question: In a concise analytical response, explain why correcting uncertain or noisy labels may be acceptable in one context but not in another.

Sample answer: If the goal is to improve a deployed system or guide product iteration, adjusting labels after reviewing mistakes can be reasonable because the main objective is better practical performance. However, if the result will be reported in a research paper or used as an unbiased estimate of model accuracy, the same adjustment becomes a problem because it can distort the measurement. In short, acceptability depends on whether the intended use can tolerate bias or requires a neutral evaluation.

Key points:

• Reasonable for product or system improvement
• Problematic for research reporting
• Problematic for an unbiased accuracy estimate
• Intended use determines whether the bias matters

Rubric: A strong response states the development-use exception, identifies the research and unbiased-evaluation cases as problematic, and explains that acceptability depends on the goal.