Evaluating Strategies for a Novel Domain
Two teams are tasked with enabling a general-purpose Large Language Model to write accurate code in a newly developed, obscure programming language.
- Team Alpha focuses exclusively on prompt engineering. They design highly detailed prompts that include the complete language syntax, style guides, and several examples of correct code, hoping the model can learn 'on the fly' from the context provided.
- Team Beta ignores prompt engineering initially. Instead, they collect a large corpus of code written in the new language and use it to continue training the base model.
Evaluate the two strategies. Which team's approach is more likely to succeed in the long run, and why? Your explanation should focus on the relationship between a model's training data and its capabilities.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Example of Prompting Failure Due to Lack of Pre-Training Data
A research team is using a large, general-purpose language model to summarize newly discovered ancient texts written in a rare, previously untranslated dialect. The team has spent weeks crafting highly detailed instructions and providing the model with several hand-translated examples within the input. Despite these efforts, the model consistently produces summaries that are nonsensical or completely unrelated to the text. What is the most likely reason for the model's failure and the most effec
Diagnosing LLM Performance in a Specialized Domain
Evaluating Strategies for a Novel Domain