Google

During the inference phase, each layer within a Transformer executes a two-step process. First, it applies a self-attention function (Attqkv) to the input, followed by a Feed-Forward Network (FFN). The outcome of this sequence is a d-dimensional vector that represents the current token while incorporating contextual information from all preceding tokens in the sequence (the "left context").

Layer-wise Processing in Transformer Inference

The architecture of a Transformer is characterized by a stack of 'L' identical layers. The computational process, which involves a self-attention mechanism followed by a Feed-Forward Network, is executed sequentially through this entire stack. The final output representation for a given input is the result generated by the topmost, or L-th, layer of the stack.

Stacked Layer Architecture and Final Output in Transformers

The formula for single-head self-attention calculates the output for a single query vector $\mathbf{q}_{i'}$ based on a set of key-value pairs. The formula is: $$ \text{Att}_{\text{qkv}}(\mathbf{q}_{i'}, \mathbf{K}, \mathbf{V}) = \text{Softmax}(\frac{\mathbf{q}_{i'}\mathbf{K}^{\text{T}}}{\sqrt{d}}) \mathbf{V} $$ In this equation, the value matrix $\mathbf{V}$ is an element of the set of real-numbered matrices with dimensions $i' \times d$, expressed as $\mathbf{V} \in \mathbb{R}^{i' \times d}$. Here, $i'$ represents the number of key-value pairs in the sequence, and $d$ is the dimension of each value vector. The overall process involves computing the dot product of the query with all keys, scaling by the square root of the key dimension, applying the Softmax function to obtain attention weights, and finally, computing a weighted sum of the value vectors.

Formula for Single-Head Self-Attention

Within a single layer of a Transformer model during inference, a sequence of input vectors is processed through a two-step sequence. Which statement best analyzes the distinct roles of the self-attention mechanism and the subsequent Feed-Forward Network (FFN) in this process?

Arrange the following computational steps in the correct order as they occur within a single layer of a Transformer model during inference.

Given the following scenario, identify which of the two main computational components within a single Transformer layer is most likely malfunctioning and explain your reasoning.

Learn Before

Related