Forward Propagation Formulation

Backward Propagation Formulation

Dimension of weight matrix

Connection between the Layers of Neural Network

Forward Propagation Formulation

True/False: During forward propagation, in the forward function for a layer ll you need to know what is the activation function in a layer (Sigmoid, tanh, ReLU, etc.). During back propagation, the corresponding backward function also needs to know what is the activation function for layer ll, since the gradient depends on it.

Which of these is a correct vectorized implementation of forward propagation for layer $\ell$, where 1≤$\ell$≤$\mathcal{L}$?

• Z$^{[\ell]}$ = W$^([\ell])$A$^([\ell])$ + b$^([\ell])$