Learn Before
Evaluating a Chatbot's Fidelity
A customer support chatbot is designed to answer user questions based only on the contents of a specific product's user manual. The manual for a new smart camera details its setup process, technical specifications, and troubleshooting for connectivity issues. A user asks the chatbot, 'How does this camera's image quality compare to the previous model?' The provided manual does not mention any other models or make any comparisons. The chatbot responds: 'I cannot answer your question. The provided user manual does not contain a comparison of this camera's image quality with previous models.' Analyze this response. Is it the correct behavior for the chatbot given its design constraints? Explain why or why not.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A user asks a language model, 'What is the capital of Australia?'. The model is instructed to answer only using the information from the following text: 'Sydney is a vibrant city in Australia, famous for its opera house and harbor bridge. It is the most populous city in the country and a major hub for tourism and finance.' Which of the following is the most appropriate and faithful response for the model to generate?
Evaluating a Chatbot's Fidelity
Generating a Faithful LLM Response