Ablation experiments evaluated on the English-to-German development set (newstest2013) demonstrated the specific contributions of key Transformer components, holding computational budget constant where appropriate.

Varying the number of attention heads showed that multi-head attention clearly outperforms single-head attention; a single head ($h = 1$) achieved 24.9 BLEU, which is 0.9 BLEU lower than the 8-head configuration (25.8 BLEU). However, performance degraded when the number of heads grew excessively large (such as $h = 32$ with $d_k = 16$). Furthermore, reducing the key dimension $d_k$ degraded translation performance, suggesting that learning compatibility across representations requires adequate dimensionality.

Investigating network capacity showed that larger model parameters ($d_{model} = 1024$, $d_{ff} = 4096$) consistently improved performance. Regularization also proved critical: dropping dropout entirely ($P_{drop} = 0.0$) reduced performance to 24.6 BLEU, demonstrating its necessity in avoiding overfitting. Finally, replacing sinusoidal positional encodings with learned positional embeddings yielded nearly identical development performance (25.7 BLEU vs. 25.8 BLEU).