My derivation of CNN backpropagation formula.page 2

So I tried to derive the formula of the back-propagation my self. Here is my derivation. So suppose we have the input I with the shape H x H x C, where B - the batch, H - is the height and width of the image( just taking square image fo rate example), C - the amount of channels. The kernel k has a size of K x K x C( also taking square kernel for the sake of the example). page 1

University of California, Berkeley

The back-propagation in CNN is done on the same principle of the chain rule, but the the the way it is actually done is not that easy. Of course if we have the gradient of the output we could do bunch of nested loops and calculate the gradients of the parameters and inputs, but this would be crazy inefficient. We need to search a way to vectorize this. We already implemented the forward propagation, and convolution operation is heavily involved in back propagation. The formulas for the gradients of the input are the following:(Suppose L is the cost function, F is the current filter, X is the current input)


Backpropagation in CNN

A very good source on how Convolution Neural Networks work mathematically:
https://www.jefkine.com/general/2016/09/05/backpropagation-in-convolutional-neural-networks/


Backpropagation In Convolutional Neural Networks

Good article for buliding an intuition on how to derive the backpropagation formula:https://medium.com/@pavisj/convolutions-and-backpropagations-46026a8f5d2c

Learn Before

Related

Learn After