Categories
Artificial Intelligence (AI) Mastering Development

How do I calculate the partial derivative with respect to $x$?

I am trying to implement CNN using python Numpy.

I searched so much, but all I found was for one filter with one channel for Convolution.

Suppose we have an X as Image with this shape: (N_Height, N_Width, N_Channel) = (5,5,3)

And Let’s say I have 16 filters with this shape: (F_Height, F_Width, N_Channel) = (3,3,3) , stride=1 and padding=0

Forward:

Output shape after conv2d will be

(
math.floor((N_Height - F_Height + 2*padding)/stride + 1 )),
math.floor((N_Width- F_Width + 2*padding)/stride + 1 )),
filter_count
)

So the output of this layer will be an array with this shape: (Height, Width, Channel) = (3, 3, 16)

BackPropagation:

Suppose $dL/dh$ is the input for my layer in backpropagation with this shape: (3,3,16)

Now I must find $dL/dw$ and $dL/dx$: $dL/dw$ to update my filters params and $dL/dx$ to pass it as input to the previous layer as Loss respect to the input X.

From this answer Error respect to filters weights I found how to calculate $dL/dw$.

The problem I have in BackPropagation is I don’t know how to calculate $dL/dx$ having this shape:(5,5,3) and pass it to the prev layer.

I read lots of articles in Medium and other sites but I don’t get how to calculate it:

How Backpropagation works in a CNN
The best explanation of Convolutional Neural Networks on the Internet!
Backpropagation In Convolutional Neural Networks
How to propagate error back to previous layer in CNN?

Thanks in advance 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *