Question: In a short essay, explain the gap between the theoretical claim that unlimited scaling of model size and training data can perform very well, and the practical realities that limit such scaling.

Sample answer: In principle, if you could increase neural network size and training data without limit, you could do very well on many learning problems. In practice, however, two constraints intervene: training very large models is slow, creating computational problems, and the supply of additional training data can be exhausted. As a result, teams cannot rely purely on scaling; they must work within these computational and data-availability limits.

Key points:

• Theoretical claim: unlimited size and data can perform very well
• Practical limit 1: very large models are slow to train, causing computational problems
• Practical limit 2: available training data supply can be exhausted
• Recognition that practice diverges from the theoretical ideal

Rubric: Full credit requires stating the theoretical claim, identifying both practical limits (computation/speed and data exhaustion), and articulating why this creates a gap between theory and practice.