Critique of a Prompt for Deliberation Testing
A researcher wants to evaluate a language model's ability to produce a correct translation after being shown a source sentence and a separate, randomly chosen (and likely incorrect) translation. The primary goal is to test the model's ability to generate a good output from this context, without also testing its ability to explicitly identify and label errors.
The researcher drafts the following prompt: `Analyze the error in the provided English translation of the Chinese sentence below, and then write the correct translation.
Chinese: 这本书很有趣。 Incorrect English: This book is very interest.`
Critique this prompt based on the researcher's primary goal. How could the prompt be simplified to better isolate the model's generation capability from its error analysis capability?
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A researcher is designing an experiment to evaluate a language model's ability to produce a correct translation after being presented with a source sentence and a separate, potentially flawed translation example. The goal is to use a prompt that simply presents the source information and elicits a new translation, without explicitly instructing the model to 'correct' or 'fix' the flawed example. Given the Chinese source sentence '这道菜太咸了' (This dish is too salty), which of the following prompt structures best achieves this specific goal?
Critique of a Prompt for Deliberation Testing
You are setting up a prompt to test a large language model's deliberation-only translation capabilities. Your goal is to translate the French sentence 'Le chat est sur la table.' into English. Complete the prompt below by filling in the blank with the second line that correctly follows the standard template for this task.
Given the French sentence: Le chat est sur la table.____