Evaluating a Chatbot Development Strategy
Critique the proposed strategy in the following scenario. Based on the principles of training language models on datasets of varying sizes, what is the most likely outcome for the model's performance in each language, and why is the lead engineer's assumption flawed?
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Prep Sessions
Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Ch.2 Model Scaling and Capability Evaluation - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
Multilingual Language Understanding on MMLU - Transformer Architecture and Large Language Model Capabilities @ University of Michigan - Ann Arbor
OpenStax Psychology (2nd ed.) Textbook
Related
A company builds a single, large-scale language model by training it on a massive dataset composed of text scraped from the public internet. During testing, the model demonstrates excellent fluency and accuracy for tasks in German, but its performance in the Irish language is poor, characterized by frequent grammatical errors and irrelevant responses. What is the most probable cause for this significant difference in performance?
Evaluating a Chatbot Development Strategy
A multilingual large language model will reliably achieve uniform competence across all languages included in its training mixture.
A development team trains a multilingual large language model on text spanning dozens of languages, but discovers that the model demonstrates poor performance when processing low-resource languages. Explain why this performance disparity occurs and identify the two critical attributes of language-specific training data that determine a model's performance in that language.