# Backpropagation for jacobian confusion

I am following Pattern Recognition and Machine Learning by Christopher Bishop chapter 5 on Neural Networks and I do not fully understand the formulas for backpropagation in order to calculate the Jacobian.

Consider a Network with 2 input units $$x_{i}$$, 2 hidden units, and 2 output units $$y_{k}$$.

We want to solve:

$$J_{ki} = \frac{\partial y_{k}}{\partial x_{i}} = \sum_{j} \frac{\partial y_{k}}{\partial a_{j}} \frac{\partial a_{j}}{\partial x_{i}}$$

The sum over $$j$$ is a sum over the 2 hidden units. Hence $$\frac{\partial y_{k}}{\partial a_{j}}$$ measures the change in output $$y_{k}$$ due to a change in the activation $$a_{j}$$ of hidden unit $$j$$ and $$\frac{\partial a_{j}}{\partial x_{i}}$$ measures the change in the activation $$a_{j}$$ due to a change in the input. As $$a_{j} = \sum_{i} w_{ji}x_{i}$$, $$\frac{\partial a_{j}}{\partial x_{i}}$$ is clearly $$w_{ji}$$.

Hence we have:

$$J_{ki} = \sum_{j} \frac{\partial y_{k}}{\partial a_{j}} \frac{\partial a_{j}}{\partial x_{i}} = \sum_{j} \frac{\partial y_{k}}{\partial a_{j}} w_{ji}$$

Now is where I cannot completely follow the formulas. Above we see that we want to calculate $$\frac{\partial y_{k}}{\partial a_{j}}$$ for each hidden unit. It is given by the following formula:

$$\frac{\partial y_{k}}{\partial a_{j}} = \sum_{l} \frac{\partial y_{k}}{\partial a_{l}} \frac{\partial a_{l}}{\partial a_{j}}$$

I do not understand why we are summing over $$l$$ and what $$a_{l}$$ should be. If $$a_{j}$$ is a single hidden unit and $$y_{k}$$ is a single output unit, why would we need a sum to measure the relationship between these two. Bishop says that "the sum runs overr all units $$l$$ to which $$j$$ sends connections".