Decide whether a team may accept label-correction bias in two planned uses of its result.
Case context: A team is developing an application and initially needs the result only for that development effort. Later, it considers using the same result in an academic research paper.
Question: Diagnose when the label-correction bias is acceptable and when it becomes problematic.
Sample answer: The bias is acceptable while the team's only goal is developing the application. It becomes problematic once the team plans to use the result in an academic research paper. If the team also needs a completely unbiased measure of test-set accuracy, the bias is problematic for that reason as well.
Key points:
- Development-only use permits the bias
- Academic-paper use makes the bias problematic
- A completely unbiased test-set accuracy requirement also makes it problematic
- The classification changes with the intended use
Rubric: The response must distinguish the development-only stage from the academic-use stage and correctly classify the bias in each. Mentioning the unbiased-accuracy condition strengthens the answer.
0
1
Tags
Machine Learning
Deep Learning
Supervised Learning
Dive into Deep Learning @ D2L
Data Science
Machine Learning Strategy
Machine Learning Yearning @ DeepLearning.AI
Related
In which situation is label-correction bias acceptable?
Label-correction bias is problematic when reporting results in an academic paper.
Label-correction bias is acceptable when the goal is only to develop a _____ or application.
Match each intended use with the correct judgment about label-correction bias.
Order the reasoning process for deciding whether label-correction bias is acceptable.
Explain why the acceptability of label-correction bias depends on the intended use of the result.
Decide whether a team may accept label-correction bias in two planned uses of its result.
What two needs make label-correction bias problematic?
A team needs completely unbiased test-set accuracy. How should it regard label-correction bias?
A development-only application project and an academic paper should treat label-correction bias identically.