Learn Before
Representation of a Layer's State as a Sequence of Vectors
The state of a model at a specific layer, often denoted by a capital letter like , is represented as an ordered sequence of vectors. For example, . Each bolded element is a vector corresponding to the i-th position in the sequence at layer l. This collection of vectors encapsulates the full set of representations for that layer.
0
1
Tags
Ch.3 Prompting - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Related
Matrix as a Stack of Row Vectors
Representation of a Layer's State as a Sequence of Vectors
A computational model represents the three words in the sentence 'AI models learn' as three distinct numerical vectors:
v1 = [0.1, 0.5],v2 = [0.9, 0.2], andv3 = [0.4, 0.7]. To process these words as a single, ordered sequence, how should these vectors be organized into a single matrix?Interpreting Matrix Representations in a Language Model
Analyzing a Layer's State Matrix in a Language Model
Learn After
Input Representation in a Transformer Layer
A computational model is processing a sequence of three items (at positions 0, 1, and 2). The model's internal state at its 5th processing stage is represented by the collection of vectors . What does the single vector represent in this context?
Representing a Layer's State for a Sequence
Applying Layer State Representation