When a neural network's hidden layer parameters are initialized to a constant value, standard gradient-based iteration algorithms, such as minibatch stochastic gradient descent, update the parameters uniformly and cannot break the resulting parameter symmetry on their own. However, applying dropout regularization is capable of breaking this symmetry, allowing the network to overcome the limitations of uniform weights and eventually realize its full expressive power.