Concept

Dropout Behavior at Extreme Probabilities

When evaluating a dropout layer implementation, its behavior varies predictably at boundary probabilities. Applying a dropout operation with a probability of 00 leaves the input completely unchanged, as no elements are dropped. Using a dropout probability of 0.50.5 drops approximately half of the elements and rescales the surviving ones by a factor of 22. Conversely, applying a dropout probability of 11 drops all elements entirely, resulting in a tensor of zeros.

0

1

Updated 2026-05-07

Contributors are:

Who are from:

Tags

D2L

Dive into Deep Learning @ D2L