Essay

Explain why model capacity changes the importance of training-data matching.

Question: In a concise analytical response, explain why a large mismatch between auxiliary and target data is less concerning for a sufficiently large model but more concerning under limited computation.

Sample answer: A sufficiently large and flexible model can learn properties of both the auxiliary and target distributions without the sources seriously competing for representation capacity. When the auxiliary dataset is much larger, however, modeling both sources may require substantially more computation and a much larger network. If those resources are unavailable, matching training data more closely to the dev/test distribution becomes more important. Lowering the auxiliary data's weight is a compromise that preserves its use while reducing the capacity required to perform well across both sources.

Key points:

  • A large enough model can represent both data sources.
  • Limited capacity makes dev/test distribution matching more important.
  • A much larger auxiliary dataset can raise computational requirements.
  • Lower auxiliary weighting reduces the need for a massive network while retaining the data.

Rubric: A strong response accurately contrasts sufficient and limited capacity, connects a dominant auxiliary source to increased computational demands, and explains lower auxiliary weighting as the source-supported compromise.

0

1

Updated 2026-07-20

Contributors are:

Who are from:

Tags

Machine Learning

Deep Learning

Supervised Learning

Dive into Deep Learning @ D2L

Data Science

Machine Learning Strategy

Machine Learning Yearning @ DeepLearning.AI