Case Study

Applying Human Labeling to a Medical AI Initiative

Case context: You are leading an AI team developing a machine learning model to detect anomalies in X-ray images. Your team is deciding how to acquire a large, accurate dataset to train the algorithm, as current automated heuristics are failing to produce reliable labels.

Question: Given that interpreting X-rays is a task performed well by trained professionals, what labeling strategy should you decide to implement, and what error rate might you realistically target based on Machine Learning Yearning principles?

Sample answer: I would decide to use a team of doctors as human labelers to annotate the X-ray images. Because medical professionals already perform this task well, they can provide high-accuracy labels. Based on Andrew Ng's examples, it is realistic to expect a team of doctors to provide these labels with a low error rate, such as around 2%.

Key points:

  • Employ human experts (doctors) as labelers.
  • Medical imaging is a task humans (doctors) perform well.
  • Human experts provide high-accuracy labels.
  • A team of doctors can achieve a low error rate (e.g., 2%).

Rubric: The learner must recommend using human experts (doctors) to label the data and mention that humans can achieve a low error rate (specifically citing the ~2% rate for medical imaging teams).

0

1

Updated 2026-06-13

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI