Case context: A machine learning team has finished their initial prototype. They are deciding between multiple possible improvement directions, such as collecting more data, training longer, or changing the model architecture. One engineer suggests they look for specific patterns and clues in their current model's errors before deciding, while another engineer wants to immediately start a three-month data collection campaign.

Question: Using the concepts of machine learning strategy, diagnose the two proposed approaches. Which approach should the team choose to decide on their next direction, and what is the potential impact of this decision on their development timeline?

Sample answer: The team should follow the first engineer's suggestion to look for clues in their current system's behavior before deciding on a direction. Machine learning problems leave clues that tell you what is useful and what is not useful to try. If the team chooses a direction poorly (such as embarking on the data collection campaign without checking if it is useful), they might waste months. Learning to read the clues first will save them months or years of development time.

Key points:

• Analyzing the system for clues reveals what is useful or not useful to try next.
• Choosing an improvement direction poorly (without reading clues) can waste months.
• Learning to read problem clues can save the team months or years of development time.

Rubric: The response must recommend analyzing the current system for clues first. It must explain that clues indicate what is useful or not useful to try. It must state that choosing poorly can waste months, whereas learning to read these clues can save months or years of development time.