Evaluating a Prompt Instruction for LLM Reliability
A common technique to improve the reliability of a language model is to include the following instruction in its prompt:
Please note that your answers need to be as accurate as possible and faithful to the facts. If the information provided is insufficient for an accurate response, you may simply output "No answer!".
Critically evaluate this instruction. In your response, discuss its primary strengths in promoting factual accuracy and its potential limitations or failure modes.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is configured with the instruction below. It is then given a user's question and a specific piece of context to generate an answer.
Instruction:
Please note that your answers need to be as accurate as possible and faithful to the facts. If the information provided is insufficient for an accurate response, you may simply output "No answer!".User Question: "What was the primary cause of the 2008 financial crisis?"
Provided Context: "The 2008 financial crisis was a severe worldwide economic crisis. Major financial institutions collapsed, stock markets plunged, and millions lost their jobs. The crisis led to a global recession, the effects of which were felt for many years. Governments around the world implemented large-scale bailout programs and new financial regulations in response."
Based on the provided instruction and context, what is the most likely output from the language model?
Improving AI Assistant Reliability
Evaluating a Prompt Instruction for LLM Reliability