The Transformer offers several computational and performance advantages over recurrent architectures:

• Constant sequential operations: A self-attention layer connects all positions with a constant number of sequentially executed operations, whereas a recurrent neural network (RNN) requires $O(n)$ sequential operations. In terms of computational complexity, self-attention layers are faster than recurrent layers whenever the sequence length is smaller than the representation dimensionality.
• Faster training: For translation tasks, the Transformer can be trained significantly faster than architectures based on recurrent or convolutional layers.
• State-of-the-art translation results: On both the WMT 2014 English-to-German and WMT 2014 English-to-French translation tasks, the Transformer achieved a new state-of-the-art. On the English-to-German task, the best model outperformed even all previously reported ensembles.