Learn Before
Designing a Long-Context Memory Test
A research team wants to verify a new large language model's ability to recall information from the very beginning of a long input sequence. Based on the principles of an evaluation where a model is required to replicate a portion of its input, describe a specific, controlled experimental setup they could use. Your description should detail the structure of the input text and the nature of the final instruction given to the model.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Critique of a Long-Context Evaluation Method
A researcher designs a synthetic task where a large language model is given a 20,000-word document and is then prompted to reproduce the final paragraph verbatim. While this task assesses the model's ability to recall information, what is the primary limitation of using this specific 'copy task' to draw conclusions about the model's effective long-term memory?
Designing a Long-Context Memory Test