Choosing the Right Contextual Approach for Language Tasks
Imagine you are developing two separate language-based AI systems.
- System A: A tool designed to automatically complete a user's sentence as they type it.
- System B: A tool designed to identify and suggest corrections for a grammatically incorrect word within a complete, pre-written paragraph.
For each system, determine whether it would be more effective to use a prediction model that considers only the preceding text (unidirectional context) or one that considers both preceding and succeeding text (bidirectional context). Justify your reasoning for each choice.
0
1
Tags
Ch.1 Pre-training - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Causal Language Modeling as a Special Case of Masked Language Modeling
Example of Masked Language Modeling Prediction
Consider two different approaches for training a language model to predict a specific word within a sentence.
Approach 1: The model is trained to predict the next word in a sequence, using only the words that have appeared before it.
Approach 2: The model is trained to predict a word that has been intentionally hidden, using all the other visible words in the sentence, both those that come before and after the hidden word.
If both models are tasked with predicting the word 'jumps' in the sentence 'The quick brown fox jumps over the lazy dog', which statement correctly analyzes the contextual information available to each model for this specific task?
Choosing the Right Contextual Approach for Language Tasks
Match each description of a language model's prediction task or characteristic to the type of contextual information it utilizes.