A research lab has a highly capable language model pre-trained on a maximum sequence length of 4,096 tokens. They need to adapt this model to summarize legal documents that are frequently over 100,000 tokens long. The lab has a limited budget, making extensive re-training from scratch infeasible. Which of the following adaptation strategies would be the most effective and resource-efficient for this specific scenario?
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A research lab has a highly capable language model pre-trained on a maximum sequence length of 4,096 tokens. They need to adapt this model to summarize legal documents that are frequently over 100,000 tokens long. The lab has a limited budget, making extensive re-training from scratch infeasible. Which of the following adaptation strategies would be the most effective and resource-efficient for this specific scenario?
Diagnosing a Long-Context Adaptation Failure
Critique of a Long-Context Adaptation Strategy