Learn Before
Evaluating Context Expansion for a Chatbot
A company is developing a customer support chatbot designed to handle long, multi-turn conversations. They are debating between two strategies for handling the conversation history during inference: 1) Truncating the history to only include the last few user-chatbot exchanges to minimize computational cost, or 2) Providing the full, unabridged conversation history to the model for every new response. Evaluate the potential trade-offs of the second strategy (providing the full history) in terms of response quality, computational requirements, and overall user experience. Justify your evaluation with specific examples of how this strategy might succeed or fail.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Improving Narrative Coherence in AI-Generated Stories
A developer observes that a language model is generating summaries of long articles that lack detail and miss key points. To address this, they modify the inference process to provide the model with the full, unabridged article text instead of a shorter, pre-processed version. Which statement best analyzes why this modification is likely to improve the quality of the generated summary?
Evaluating Context Expansion for a Chatbot
Few-Shot Learning in Prompting
Chain-of-Thought (COT) Prompting
Retrieval-Augmented Generation (RAG)