Case context: Your team has measured human-level performance on clear daytime warehouse images at 1%. The system will actually be used on night-time phone photos that are dark, blurry, and sometimes partially cropped. Your current model makes 6% error on this deployment data.

Question: What should your team do to determine a fair benchmark for the night-time application data?

Sample answer: The team should sample real night-time phone photos from the target distribution and have humans label them. By computing the human error rate on those specific images, they can establish the benchmark that matches the deployment setting.

Key points:

• Use images from the night-time phone-photo distribution.
• Have humans label those target-domain images.
• Measure human error on that distribution to set the benchmark.

Rubric: The response must say that humans should label the target-domain data specifically and that this is how the benchmark error rate should be measured.