Learn Before
Evaluating System Architectures for Long-Document Q&A
A company needs to build a chatbot that can answer detailed questions about its entire 500-page employee handbook. They are considering two approaches:
- Use a model designed to process the entire handbook in a single input to find answers.
- Use a model with a smaller input capacity, combined with a system that first retrieves relevant sections of the handbook and then feeds only those sections to the model to generate an answer.
Evaluate these two approaches. In your evaluation, compare their potential strengths and weaknesses regarding answer accuracy, computational cost, and the ease of updating the handbook's information.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
General Applicability of Long-Context Methods
Context Scaling for LLM Performance Improvement
Model Selection for Large-Scale Document Summarization
A development team is tasked with creating a system that can analyze and answer questions about lengthy legal documents, some of which are over 100,000 words long. When selecting a foundational language model for this task, what is the most critical architectural characteristic they should prioritize to ensure the system can effectively process the entirety of these documents at once?
Evaluating System Architectures for Long-Document Q&A
Infinite Context Encoding in LLMs
Continuous-Space Attention for Infinite Context