Learn Before
Relation
Pros and Cons of Leaky ReLU
Pros:
- Prevents dying ReLU problem—this variation of ReLU has a small positive slope in the negative area, so it does enable backpropagation, even for negative input values
- Computationally efficient—allows the network to converge very quickly
- Non-linear—although it looks like a linear function, Leakly ReLU has a derivative function and allows for backpropagation.
Cons:
- Results not consistent—leaky ReLU does not provide consistent predictions for negative input values.
0
1
Updated 2020-10-21
Tags
Data Science