Case Study

Designing a Safe Image Filter Metric for a Public Platform

Case context: You are developing a child-friendly content filter for an image sharing application. The current evaluation metric treats a standard misclassification (e.g., misclassifying a cat as a dog) and an unacceptable error (e.g., letting through a pornographic image) with equal weight. Consequently, the team is selecting models that occasionally let through inappropriate content because their overall accuracy is high.

Question: Based on the concept of changing a metric to penalize unacceptable errors, what decision should you make regarding the evaluation metric to resolve this issue?

Sample answer: You should modify the evaluation metric to heavily penalize the unacceptable error of letting through pornographic images. Instead of treating all misclassifications equally, you should introduce a high weight or penalty factor specifically for instances where inappropriate content is classified as safe, ensuring that any model committing this error is penalized severely and not selected for deployment.

Key points:

  • Identify that the evaluation metric fails to reflect the severity of different error types.
  • Propose changing the metric by introducing a heavy penalty specifically for pornographic images.
  • Explain that the modified metric will prevent the selection of models that permit unacceptable errors.

Rubric: The answer must identify the need to change the evaluation metric, specify that a heavy penalty must be applied to the unacceptable error (letting through pornographic images), and explain how this penalty guides the model selection process away from models that leak inappropriate content.

0

1

Updated 2026-05-26

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Machine Learning Strategy

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Related