Case Study

Strategic Model Development Decision

As the lead engineer at a research lab, you must recommend a strategy for developing your next-generation language model. Your team has consistently observed that when plotting test loss versus the number of model parameters on a log-log scale, the results form a clear, straight, downward-sloping line. You have the budget for one of two options:

  1. Scaling Up: Build a new model with significantly more parameters, following the established trend.
  2. Architectural Innovation: Use the same number of parameters as your last model but invest the budget in a completely new, experimental architecture that has no performance guarantee.

Based solely on the predictable performance trend observed in your previous models, which option presents a more reliable path to achieving a lower test loss? Justify your reasoning by referencing the implications of the straight-line trend on the log-log plot.

0

1

Updated 2025-10-04

Contributors are:

Who are from:

Tags

Ch.2 Generative Models - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Evaluation in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science