Question: Explain how today’s access to large training sets and high-capacity neural networks has altered the classic bias-variance relationship. Include concrete ways to lower bias and lower variance separately.

Sample answer: In older machine learning settings, reducing bias usually came with a rise in variance, and reducing variance often meant accepting more bias. Modern deep learning is less constrained by that pattern because very large data sets and large neural networks make the tradeoff less rigid. A practitioner can often cut bias by expanding the network and tuning regularization so the model remains well behaved. At the same time, variance can often be reduced on its own by adding more training examples, without necessarily harming the model’s fit.

Key points:

• Large data sets and large networks weaken the classic tradeoff.
• Bias can often be lowered by increasing model capacity.
• Regularization helps control variance when using larger models.
• Variance can often be lowered independently by adding data.

Rubric: The essay should contrast the older bias-variance view with the modern deep learning setting, and it must mention using larger networks with appropriate regularization to reduce bias, as well as using more data to reduce variance.