Case Study

Decide whether a team may accept label-correction bias in two planned uses of its result.

Case context: A team is developing an application and initially needs the result only for that development effort. Later, it considers using the same result in an academic research paper.

Question: Diagnose when the label-correction bias is acceptable and when it becomes problematic.

Sample answer: The bias is acceptable while the team's only goal is developing the application. It becomes problematic once the team plans to use the result in an academic research paper. If the team also needs a completely unbiased measure of test-set accuracy, the bias is problematic for that reason as well.

Key points:

  • Development-only use permits the bias
  • Academic-paper use makes the bias problematic
  • A completely unbiased test-set accuracy requirement also makes it problematic
  • The classification changes with the intended use

Rubric: The response must distinguish the development-only stage from the academic-use stage and correctly classify the bias in each. Mentioning the unbiased-accuracy condition strengthens the answer.

0

1

Updated 2026-07-20

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI