Learn Before
Relation

Pros and Cons of Leaky ReLU

Pros:

  • Prevents dying ReLU problem—this variation of ReLU has a small positive slope in the negative area, so it does enable backpropagation, even for negative input values
  • Computationally efficient—allows the network to converge very quickly
  • Non-linear—although it looks like a linear function, Leakly ReLU has a derivative function and allows for backpropagation.

Cons:

  • Results not consistent—leaky ReLU does not provide consistent predictions for negative input values.

0

1

Updated 2020-10-21

Tags

Data Science