Learn Before
Data Source Selection for a Specialized LLM
A development team is building a language model specifically to help software engineers by generating high-quality technical documentation. Given the case study below, identify the TWO most critical data sources for pre-training this specialized model and justify your choices. Your justification should explain how each selected source directly contributes to the model's intended function.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Evaluating Data Sources for LLM Pre-training
Data Source Selection for a Specialized LLM
A newly developed large language model demonstrates high fluency and generates grammatically perfect, conversational text. However, it frequently provides outdated information, struggles to generate well-structured, long-form content like reports, and often fabricates details when asked about events from the last year. Based on these specific performance characteristics, which of the following descriptions most likely represents the composition of its pre-training dataset?
GPT-3
Falcon
LLaMA2
PaLM-450B
Gemma-7B