Formula

Concatenated Weight Matrix (Wh\mathbf{W}_h)

The matrix Wh\mathbf{W}_h is formed by the horizontal concatenation of a sequence of MM sub-matrices or vectors, denoted as Wh1,Wh2,,WhM\mathbf{W}_h^1, \mathbf{W}_h^2, \dots, \mathbf{W}_h^M. This construction is formally represented by the formula: Wh=[Wh1Wh2WhM]\mathbf{W}_h = \begin{bmatrix} \mathbf{W}_h^{1} & \mathbf{W}_h^{2} & \dots & \mathbf{W}_h^{M} \end{bmatrix}.

Image 0

0

1

Updated 2026-04-21

Contributors are:

Who are from:

Tags

Ch.2 Generative Models - Foundations of Large Language Models

Foundations of Large Language Models

Foundations of Large Language Models Course

Computing Sciences