Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this community
Here is the expression to take the derivative of. $$C = \frac{1}{2}\sum_j (y_j - a_j^L)^2$$
Here is the result. $$\frac{\partial C}{\partial a_j^L} = 2(a_j^L-y_j)$$
Multiplying by 2, then again by the derivative of the inside (-1) seems reasonable, but what happened to the summation?
Let's save the index $j$ for the derivative and write $$ C = \frac{1}{2} \sum_{i=1}^n (y_i - a_i^L)^2 $$ Therefore by the sum rule and chain rule, $$ \frac{\partial C}{\partial a_j^L} = \sum_{i=1}^n 2(y_i - a_i^L)(-1)\frac{\partial a_i^L}{\partial a_j^L} $$ Assuming the variables $\left(a_1^L,\dots,a_n^L\right)$ are independent, the derivative of any one of them with respect to any other one of them is zero. But the derivative of each of them with respect to itself is one. In other words, $$ \frac{\partial a_i^L}{\partial a_j^L} = \delta_{ij} = \begin{cases} 1 & i = j \\ 0 & i \neq j \end{cases} $$ The symbol $\delta_{ij}$ is called the Kronecker delta.
Returning to the summation, we see that each term is multiplied by $0$ except the case $i=j$. So the only surviving term is that one: $$ \frac{\partial C}{\partial a_j^L} = 2(y_j - a_j^L)(-1) = 2(a_j^L-y_j) $$
The derivative is with respect to a component $a_j^L$ thus the others cancel out.
The notation is more than a little confusing, but I think this is just computing the derivative with respect to a variable that appears in just one of the summands. See what it says with $j=1$ in the answer, leaving the dummy index $j$ in the summation. Or change the dummy $j$ to $i$.
