Increasing Model Capacity as a Strategy for Long Contexts
A common strategy to address the difficulty of processing long contexts is to simply increase the capacity of the memory model. This approach ensures that more contextual information can be stored and accessed, directly counteracting the limitations of low-capacity models when dealing with extensive sequences.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Learn After
A team is developing a language model to write detailed summaries of entire novels. They observe that the model's summaries are often incoherent and fail to mention key characters and events from the first few chapters. To solve this, the team wants to apply a strategy that directly increases the model's capacity to handle the full context. Which of the following actions best implements this specific strategy?
Evaluating the Strategy of Increasing Model Capacity
Troubleshooting a Chatbot's Memory