Which of the following statements is true about layers of a deep neural network?

- Single layer NN
- 2-layer NN
- N-layer NN

Match the hidden layers to the neural networks.

Consider the following neural network. How many layers does this network have?

We don't count the input layer and the number of layers is defined as the number of hidden layers plus one for the output layer.
For example, in the following MLP, there are two layers.

University of Michigan - Ann Arbor

In the chain structure example $f (x) = f ^{(3)}(f^{(2)}(f^{(1)}(x)))$, $f^{(1)}$ is called
the first layer of the network, f^{(2)} is called the second layer, and so on. The final layer of a feedforward network is called the output layer. 

During neural network training, we drive $f(x)$ to match $f^∗(x)$. The training data provides us with noisy, approximate examples of $f^∗(x)$ evaluated at different training points. Each example x is accompanied by a label $y ≈ f^*(x)$.

The training examples specify directly that the output layer must produce a value that is close to $y$ at each point $x$. However, the behavior of the other layers is not directly specified by the training data. The training data do not say what each individual layer should do. Instead, the learning algorithm must decide how to use these layers to best implement an approximation of $f^*$. Because the training data does not show the desired output for each of these layers, they are called hidden layers.

Layers of a Feed Forward Neural Network

In the chain structure of a feedforward network, Let's assume the input example is $x \in \mathbb{R}^d$, then the first layer is given by
$h^{(1)} = g^{(1)} (W^{(1)T}x + b^{(1)})$
the second layer is given by 
$h^{(2)} = g^{(2)} (W^{(2)T} h^{(1)}+ b^{(2)})$
and so on.
For the output layer,
$\hat{y} = g^{(n)}(W^{(n)T} h^{(n-1)}+ b^{(n)})$

where the matrix $W^{(k)}\in \mathbb{R}^{d\times h_k}$ is the weight parameter for layer $k$ and $g^{k}$ is the activation function for layer $k$. Whether transposing $W^{(k)}$ depends on the shape of the vector.

Connection between the Layers of Neural Network

Number of Layers in a Feed Forward Neural Network

There is a lot of research going on activation functions, new units/activation functions which vividly exhibit great performance  will get published in research papers. The common hidden units found in advanced deep learning models are. 

- Linear Hidden Units
- Softmax units
- Radial basis functions
- Softplus
- Hard tanh

Learn Before

Related

Learn After