Case context: A team building a large neural network believes that simply making the network bigger and gathering more training data will always improve performance, following the simple bias/variance remediation formula. They plan to keep doubling model size and data collection indefinitely.

Question: Based on the source, what practical limitations should this team anticipate as they try to scale indefinitely, and how might these limitations affect their plan?

Sample answer: The team should anticipate that while increasing size and data can in principle keep improving performance, in practice training very large models becomes slow and computationally expensive. They may also run out of additional training data to collect. These constraints mean the team cannot rely on indefinite scaling and must plan for computational costs and finite data availability rather than assuming unlimited scaling is feasible.

Key points:

• Recognizes the theoretical claim about unlimited scaling
• Identifies computational problems from training very large models
• Identifies finite supply of additional training data
• Connects these limits to the feasibility of the team's plan

Rubric: Full credit requires identifying both the computational/speed limit and the data exhaustion limit, and connecting them to the team's indefinite scaling plan.