Diagnosing and Improving Context Management in a Conversational AI
A developer is creating a technical support chatbot. To maintain conversation history, their current strategy is to append the entire transcript of the ongoing conversation to every new user query before sending it to the language model. They observe that while the bot performs well in short conversations, its responses become less relevant and it starts to 'forget' details mentioned earlier as conversations exceed 20-30 turns. Based on the principles of strategic information management for large context windows, analyze the two most likely reasons for the chatbot's declining performance in longer conversations. For each reason, propose a specific, alternative strategy to structure or select the conversational history that would mitigate the issue.
0
1
Tags
Ch.5 Inference - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Optimizing a Chatbot's Information Retrieval
A developer is building a legal document summarization tool using a large language model. To provide comprehensive context, for every summarization request, they prepend the full text of the three longest, most cited legal precedents related to the document's general topic. However, they find that the model's summaries often miss key nuances from the specific document being summarized and over-emphasize general principles from the provided precedents. Which of the following best explains this failure in performance?
Diagnosing and Improving Context Management in a Conversational AI