In the general case what is the dimension of $$W^{[l]}$$, the weight matrix associated with layer ll? (hint: $$n^{[l]}$$ is the number of inputs in l layer)

University of Michigan - Ann Arbor

$L$ = # of layers
$m$ = # of training datapoints
$$n^{[l]} = $$ # of units (neurons) in layer $l$
$$A^{[l]} = g^{[l]}(Z^{[l]}) = $$ activations (outputs) in layer $l$
$Z^{[l]}, A^{[l]}: (n^{[l]},m)$
$$X = A^{[0]}$$      $$\hat{Y} = A^{[L]}$$
$$W^{[l]} = $$ Weights for $$Z^{[l]}$$      $$W^{[l]}: (n^{[l]}, n^{[l - 1]})$$
$$b^{[l]} = $$ Biases for $$Z^{[l]}$$      $$b^{[l]}: (n^{[l]}, 1)$$

Formulation of a Feedforward Neural Networks

https://www.coursera.org/learn/neural-networks-deep-learning?specialization=deep-learning

Neural Networks and Deep Learning

Input: $$A^{[l-1]}$$
Output: $$A^{[l]}$$      Cache: $$Z^{[l]}, A^{[l-1]}$$
$$Z^{[l]} = W^{[l]} A^{[l-1]} + b^{[l]}$$
$$A^{[l]} = g^{[l]}(Z^{[l]})$$

Forward Propagation Formulation

Input:

$$dA^{[l]}$$

   For the last layer (L): 

$$dA^{[L]} = \sum_{i=1}^{m} -\frac{y^{[1]}}{a^{[1]}} + \frac{1 - y^{[1]}}{1 - a^{[1]}}$$

Output: 

$$dA^{[l-1]},   dW^{[l]},   db^{[l]}$$

$$dZ^{[l]} = dA^{[l]} g'^{[l]}(Z^{[l]})$$

$$dW^{[l]} = \frac{1}{m} dZ^{[l]} A^{[l - 1] T}$$

$$db^{[l]} = \frac{1}{m} \sum dZ^{[l]}$$

$$dA^{[l - 1]} = W^{[l]T} dZ^{[l]}$$ 

Learn Before

Related