Essay

Evaluating Inference-Time Scaling Strategies

A research lab is developing a medical diagnosis assistant using a large language model. They are considering two inference-time strategies.

Strategy A: Use the single largest available model with a very long context window to process the entire patient history at once, aiming for the most comprehensive single analysis.

Strategy B: Use a smaller, faster model but run it multiple times with different prompts that focus on different aspects of the patient's history (e.g., symptoms, lab results, family history), then have a separate aggregation model synthesize the results.

Evaluate which strategy better embodies the broader definition of inference-time scaling, justifying your answer by referencing concepts like robustness and exploration.

0

1

Updated 2025-10-02

Contributors are:

Who are from:

Tags

Ch.5 Inference - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences

Evaluation in Bloom's Taxonomy

Cognitive Psychology

Psychology

Social Science

Empirical Science

Science