Learn Before
Calculating Transformation Matrix Dimensions
In a neural network component, the transformation matrix for each of its parallel processing units is defined by the formula , where is the model's embedding dimension and is the number of parallel units. If a model has an embedding dimension () of 768 and uses 12 parallel units (), what are the dimensions of a single transformation matrix ? Show your calculation.
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Application in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
In a neural network component, a transformation matrix for each of several parallel processing units is defined by the formula , where is the model's primary data representation dimension and is the number of parallel units. If a system architect decides to double the number of parallel units () while keeping the primary dimension () constant, what is the resulting effect on the dimensions of the matrix for each individual unit?
Analysis of Transformation Matrix Dimensions
Calculating Transformation Matrix Dimensions