– For small data NN can perform as Linear regression or SVM (Support vector machine)

– For big data a small NN is better that SVM

– For big data a big NN is better that a medium NN is better that small NN.

– Hopefully we have a lot of data because the world is using the computer a little bit more

    - Mobiles

    - IOT (Internet of things)

University of Michigan - Ann Arbor

• Single neuron == linear regression without applying activation(perceptron)

• Basically a single neuron will calculate weighted sum of input(W.T*X) and then we can set a threshold to predict output in a perceptron. If weighted sum of input cross the threshold, perceptron fires and if not then perceptron doesn’t predict.

What is a Neural Network(NN)?

• Hidden layers predicts connection between inputs automatically, thats what deep learning is good at.

• Deep NN consists of more hidden layers (Deeper layers)

• Each Input will be connected to the hidden layer and the NN will decide the connections.

Simple NN and Deep NN

Scale drives deep learning progress

• The main steps for building a Neural Network are:

     – Define the model structure (such as number of input features and outputs)

     – Initialize the model’s weights and biases.

     – Loop.

           ∗ Calculate current loss (forward propagation)

           ∗ Calculate current gradient (backward propagation)

           ∗ Update parameters (gradient descent)

• Preprocessing the dataset is important.

• Tuning the learning rate (which is an example of a “hyperparameter”) can make a big difference to the algorithm.

Main Step to build a NN

It is easy to build a perceptron that computes the logical functions OR and AND, however it is impossible to build one that computes logical XOR. This is because the perceptron is a linear classifier which acts as a decision boundary in two-dimensional space. 

Whilst OR and AND are linearly separable functions, XOR is not, it is impossible to draw a straight line that correctly separates its two categories. As shown in the figures below where $x_1$ and $x_2$ are inputs into the logical functions; filled circles represent outputs of 1, and white circles outputs of 0.

Learn Before

Related