Question: Compare evaluating a new spam filter by manually trying it in the product with evaluating it using a validation set and a single metric. Explain how each approach affects the ability to notice small gains and how it changes the pace of development.

Sample answer: If a team relies on manual product testing, every candidate spam filter must be inserted into the live system and checked by hand. That process takes a long time and makes subtle improvements hard to notice. By contrast, a validation set and one clear metric let the team score each candidate automatically and quickly. This makes it much easier to spot both small and large gains, so the team can keep promising ideas and drop weak ones sooner. The result is a much faster development cycle.

Key points:

• Manual testing requires putting each candidate into the product and checking it by hand, which is slow.
• A validation set and single metric provide fast, automatic comparison across models.
• Faster feedback makes it easier to detect small improvements and decide what to keep or discard.

Rubric: The response should contrast slow manual product testing with fast validation-set evaluation. It must explain that manual testing makes small improvements difficult to detect, while a validation set and metric help the team identify improvements quickly and iterate faster.