Without activation functions between layers, the result of the calculus is just the linear combination of  weights, which is not different from a single layer network


University of Colorado at Boulder

Input: $$A^{[l-1]}$$
Output: $$A^{[l]}$$      Cache: $$Z^{[l]}, A^{[l-1]}$$
$$Z^{[l]} = W^{[l]} A^{[l-1]} + b^{[l]}$$
$$A^{[l]} = g^{[l]}(Z^{[l]})$$

Forward Propagation Formulation

An on-going but a helpful book resource about NLP
https://web.stanford.edu/~jurafsky/slp3/

Learn Before

Related