Case context: A healthcare startup trains two predictors on the same patient-record dataset. One is a simple linear classifier, and the other is a large neural network. The training set grows from 12,000 records to 3.2 million records. The linear classifier improves a little at first and then levels off, while the neural network keeps getting better as more records and more GPU training time are added.

Question: Using the idea that scale drives progress in deep learning, explain why the two models respond differently as the dataset grows.

Sample answer: The linear classifier reaches a point where extra records do not help much because its representational power is limited. In contrast, the large neural network can keep extracting value from larger datasets, so its performance continues to rise as more labeled examples become available. That improvement, however, only happens if the team also has enough compute to train the bigger model on the larger dataset. The example shows that progress comes from both more data and the computational resources needed to use that data effectively, which is why the deep network benefits from scale while the simpler model stalls.

Key points:

• The simpler model levels off because it cannot use much more data effectively
• The neural network keeps improving as data increases
• Compute is needed to train the larger model on the larger dataset
• The example illustrates how data scale and compute together drive progress

Rubric: Full credit identifies that the simpler model plateaus because of limited capacity, that the neural network can continue improving with more data, and that sufficient compute is required to realize that benefit. Partial credit for mentioning only the plateau or only the neural network's continued gains without linking both scale factors.