The Transformer model presents several computational and performance advantages:

• A self-attention layer connects all positions with a constant number of sequentially executed operations, whereas a recurrent neural network (RNN) requires $O(n)$ sequential operations. In terms of complexity, self-attention layers are faster than recurrent layers when the sequence length is smaller than the representation dimensionality.
• For translation tasks, the Transformer can be trained significantly faster than architectures based on recurrent or convolutional layers.
• On both WMT 2014 English-to-German and WMT 2014 English-to-French translation tasks, the Transformer achieved a new state-of-the-art. In the former task, the best model outperformed even all previously reported ensembles.