Learn Before
Trade-offs of Model Depth
A research team is developing a new language model. They are deciding between a version with 12 layers and a version with 48 layers. Describe one significant advantage and one significant disadvantage of choosing the 48-layer version over the 12-layer version.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
BERT's Core Architecture
Output Probability Calculation in Transformer Language Models
Trade-offs of Model Depth
An AI team is developing solutions for two distinct tasks: Task A, which involves classifying short customer reviews as positive or negative, and Task B, which requires generating concise summaries of long, complex legal documents. They have two available models: Model X with 6 stacked processing layers and Model Y with 24 stacked processing layers. Based on the relationship between model depth and capability, which of the following strategies is most appropriate?
Analyzing the Impact of Increasing Model Layers