Learn Before
A language model is processing the sentence: 'The delivery driver carefully parked the large van.' To understand the context of the word 'parked', the model generates a 'focus' vector for it. This 'focus' vector is then compared against a 'label' vector for every word in the sentence to calculate relevance scores. What is the primary function of the 'label' vector associated with the word 'van' in this process?
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Analysis in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
A language model is processing the sentence: 'The delivery driver carefully parked the large van.' To understand the context of the word 'parked', the model generates a 'focus' vector for it. This 'focus' vector is then compared against a 'label' vector for every word in the sentence to calculate relevance scores. What is the primary function of the 'label' vector associated with the word 'van' in this process?
Diagnosing Attention Mechanism Failure
Consequences of Non-Unique Identifiers in Attention