Diagnosing a Faulty Q&A System Prompt
A company's internal Q&A bot is designed to answer employee questions using only official company documents. However, employees have noticed that the bot sometimes provides answers based on its general, pre-existing knowledge, leading to outdated or incorrect information. Below is the typical structure of the message sent to the language model:
User Question: [The employee's question is inserted here.] Retrieved Context: [Relevant text from an official company document is inserted here.]
Based on the standard components of a complete and effective prompt for such a system, identify the key component that is missing and explain how its absence leads to the observed problem.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Example Prompt Instruction for Faithfulness and Abstention
A developer is constructing a message to send to a large language model to answer a user's question based on a retrieved document. The developer has the following three components:
- The Instruction:
You are a helpful assistant. Answer the user's question based *only* on the provided text. If the information is not in the text, state that the answer cannot be found. - The User's Question:
What is the boiling point of ethanol? - The Retrieved Context:
Ethanol, a volatile, flammable, colorless liquid, has a boiling point of 78.37 °C.
Which of the following arrangements of these components creates the most effective and logically structured message for the model to process and follow the instructions correctly?
- The Instruction:
Diagnosing a Faulty Q&A System Prompt
Task Definition and Grounding in RAG Prompts
Constructing a RAG Prompt