Question: Explain why human performance is a weak reference point for a task such as forecasting hourly electricity demand, while it is a much stronger reference point for a task such as identifying objects in photos.

Sample answer: When people are not naturally good at a task like electricity-demand forecasting, their performance does not provide a reliable benchmark. First, it is harder to create trustworthy labels because humans cannot confidently tell which forecast is best. Second, human intuition is less useful for finding and diagnosing mistakes, since there is no expert judgment to lean on in a task most people handle poorly. Third, it is difficult to know what error level should be considered acceptable or close to the practical limit. By contrast, in a task where people perform very well, human results give a much clearer standard for labels, debugging, and setting a target error rate.

Key points:

• A weak human baseline makes label quality harder to judge
• It gives little help when trying to understand model errors
• It is hard to estimate a realistic best-possible error rate
• Human-strong tasks provide a more informative benchmark for all three

Rubric: Full credit: explains all three limitations of a weak human baseline—labeling difficulty, poor use of human intuition for error analysis, and uncertainty about the best achievable error rate—and contrasts them with tasks where humans are strong. Partial credit: mentions one or two of the limitations or gives only a vague comparison without explaining why the baseline is weaker.