Learn Before
Alternative Dimensions of LLM Scaling
While scaling Large Language Models is often associated with increasing model size and data volume, enhancement can also be pursued in other directions. A key alternative is adapting models for new capabilities, such as processing input sequences that are substantially longer than those encountered during their initial training.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Ch.3 Prompting - Foundations of Large Language Models
Related
Alternative Dimensions of LLM Scaling
Large-Scale Pre-training for LLMs
A development team is working on enhancing their company's language model. They are considering two different projects. Project Alpha involves training a new, much larger model from scratch on a petabyte-scale dataset to create a more powerful and knowledgeable general-purpose assistant. Project Beta involves modifying their existing model to enable it to accurately summarize entire books, which requires processing text inputs that are hundreds of times longer than what it can currently handle.
Large-Scale Pre-training of LLMs
LLM Strategy for a Financial Tech Startup
Match each primary strategy for scaling Large Language Models with its corresponding description and goal.
Learn After
Architectural Adaptation of LLMs for Long Sequences
Types of LLM Scaling
Multifaceted Nature of LLM Scaling
Inference-Time Compute Scaling for Improved Reasoning
A research lab has a powerful language model that is highly effective at generating short, creative story paragraphs. The lab now wants to use this model to write entire multi-chapter novels, which requires maintaining plot consistency and character arcs over tens of thousands of words. Which of the following development priorities best represents a shift in scaling dimension to meet this new requirement?
Evaluating a Model Scaling Strategy
Scaling LLMs Beyond Size