Formula

Vector Multiplication over Concatenated Matrices

When a vector h\mathbf{h} is multiplied by a matrix Wh\mathbf{W}_h that is formed by the horizontal concatenation of several sub-matrices (Wh1,Wh2,,WhM\mathbf{W}_h^1, \mathbf{W}_h^2, \dots, \mathbf{W}_h^M), the operation is distributive. This means the result is equivalent to a new matrix formed by concatenating the products of the vector h\mathbf{h} with each individual sub-matrix. This property is mathematically expressed as: hWh=h[Wh1Wh2WhM]=[hWh1hWh2hWhM]\mathbf{h} \mathbf{W}_h = \mathbf{h} \begin{bmatrix} \mathbf{W}_h^{1} & \mathbf{W}_h^{2} & \dots & \mathbf{W}_h^{M} \end{bmatrix} = \begin{bmatrix} \mathbf{h} \mathbf{W}_h^{1} & \mathbf{h} \mathbf{W}_h^{2} & \dots & \mathbf{h} \mathbf{W}_h^{M} \end{bmatrix}.

Image 0

0

1

Updated 2026-04-21

Contributors are:

Who are from:

Tags

Ch.2 Generative Models - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences