Question: A team notices that a classifier fits the training set too closely. Compare reducing the model's capacity with adding regularization as ways to limit overfitting. What trade-offs matter, and when is the smaller model actually the better choice?

Sample answer: Reducing capacity by cutting layers, units, or other parameters can lower variance, but it usually raises bias because the model has less flexibility. In many cases, regularization is the better first choice because it often improves generalization without giving up as much representational power. The main practical advantage of a smaller network is lower computation: it uses less memory, trains faster, and can be easier to deploy. So shrinking the model makes sense mainly when speed, cost, or deployment limits are more important than squeezing out the best accuracy.

Key points:

• Smaller models can reduce variance but often increase bias.
• Regularization is usually preferred when the goal is stronger generalization.
• The main benefit of a smaller model is lower compute and faster training.
• Choose model shrinkage mainly when efficiency constraints matter.

Rubric: The response must explain that reducing model size can reduce variance while increasing bias. It must compare this option with regularization and note that regularization usually gives better generalization performance. It must also identify faster training and lower computational cost as the main advantage of a smaller model, making it appropriate when efficiency is the priority.