Is it expected that adding an additional hidden layer to my 3-layer ANN reduces accuracy significantly?

I’ve been using several resources to implement my own artificial neural network package in C++. Among some of the resources I’ve been using are https://www.anotsorandomwalk.com/backpropagation-example-with-numbers-step-by-step/ https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ https://cs.stanford.edu/people/karpathy/convnetjs/intro.html, as well as several others. My code manages to replicate the results in the first two resources exactly. However, these are fairly simple networks in terms of depth.…