Insufficiency of Prompting Without Foundational Knowledge
Prompting and in-context learning are ineffective if a Large Language Model lacks the necessary foundational knowledge from its pre-training phase. When a model has not been trained on relevant data for a specific domain or language, even the most well-designed prompts cannot elicit a good performance. In such scenarios, the appropriate solution is to continue training the model on the required data rather than attempting further prompt refinement.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Learn After
Example of Prompting Failure Due to Lack of Pre-Training Data
A research team is using a large, general-purpose language model to summarize newly discovered ancient texts written in a rare, previously untranslated dialect. The team has spent weeks crafting highly detailed instructions and providing the model with several hand-translated examples within the input. Despite these efforts, the model consistently produces summaries that are nonsensical or completely unrelated to the text. What is the most likely reason for the model's failure and the most effec
Diagnosing LLM Performance in a Specialized Domain
Evaluating Strategies for a Novel Domain