
1. The total complexity per layer is reduced
2. The amount of computation that can be parallelized, as measured by the minimum number of sequential operations required also reduces.
3. Reduction of the path length between long-range dependencies in the network.


Advantages of using the model

1. This model generally has an encoder - decoder structure.
2. The encoder maps an input sequence of symbol representations to a sequence of continuous representations.
3. The decoder then generates an output sequence on symbols, one element at a time.


New York University

1. The transformer model proposed in this paper is an architecture that relies entirely on the attention mechanism to draw global dependencies b/w input and output.
2. It allows significantly more parallelization and has a huge role in improving translation quality after being trained for short periods of time on 8 P100 GPUs (12hrs).


Learn Before

Related

Learn After