Question: Explain how increasing the size of a neural network and the amount of training data changes the need for hand-designed intermediate features in an end-to-end system.

Sample answer: When a model is large enough and trained on a very large dataset, it can learn useful internal features directly from the raw input. In that setting, the system is less dependent on manually designed intermediate representations, because the network can discover the patterns it needs on its own. With enough scale, this approach can also move performance closer to the best achievable error rate.

Key points:

• Requires a sufficiently large neural network
• Requires a large amount of training data
• Reduces the need for hand-designed intermediate features
• Can help the system approach the best achievable error rate

Rubric: The answer should clearly connect larger model capacity and more training data to the ability to rely less on manual feature engineering.