Learn Before
Analyzing Language Model Outputs
A language model built with a deep neural network processes the input sequence 'The forecast calls for heavy rain, so the sky is'. The model's output indicates that the token 'overcast' has a higher probability than the token 'blue'. Explain the fundamental nature of this model's direct output that allows it to represent the likelihood of multiple different words as the next token.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Probability Distribution Formula for an Encoder-Softmax Language Model
Auto-Regressive Generation Process
Formal Definition of LLM Inference
Model Parameterization by θ
A language model built with a deep neural network is given the input sequence 'The cat sat on the'. The model's vocabulary consists of the following tokens: {a, cat, hat, mat, on, sat, the}. What does the model produce as its immediate, direct output to predict the very next token?
Analyzing Language Model Outputs
Explaining Language Model Output Behavior