Case context: A startup is training a deep neural network for a voice-command recognition system. The model shows high variance on validation data, but each training run currently takes nearly 30 hours because the team has only one modest GPU. The project must reach a usable version within two weeks, so the team needs a way to improve generalization without making experimentation even slower.

Question: Should the team reduce the model size or add regularization? Explain which choice fits the situation best and why.

Sample answer: Reducing the model size is the better choice in this case. A smaller network with fewer layers or fewer units will usually train faster and use less computation, which matters because the team is constrained by slow hardware and a tight delivery schedule. This change can also lower variance, even though it may raise bias.

Adding regularization is often the stronger general-purpose fix for high variance when compute cost is not a major issue, because it usually preserves predictive performance better than shrinking the network. However, regularization does not solve the urgent need for faster training. Since iteration speed is a major constraint here, shrinking the model is the more practical tradeoff.

Key points:

• Reducing model size can cut training time and computational cost.
• Smaller models can also reduce variance, though they may increase bias.
• Regularization is often preferred when the main goal is classification performance and training cost is acceptable.
• The deciding factor in this scenario is the need for faster experimentation under limited compute.

Rubric: A complete answer should compare both remedies for high variance and then choose the one that best fits the training constraints. It should explain that reducing model size is appropriate because it helps the team train faster, while regularization does not directly improve speed. It should also note that, when compute is not a concern, regularization is usually the better variance-reduction approach for final predictive performance.