Why is the summation dropped when calculating this derivative? Here is the expression to take the derivative of.
$$C = \frac{1}{2}\sum_j (y_j - a_j^L)^2$$
Here is the result.
$$\frac{\partial C}{\partial a_j^L} = 2(a_j^L-y_j)$$
Multiplying by 2, then again by the derivative of the inside (-1) seems reasonable, but what happened to the summation?
 A: The derivative is with respect to a component $a_j^L$ thus the others cancel out.
A: Let's save the index $j$ for the derivative and write
$$
    C = \frac{1}{2} \sum_{i=1}^n (y_i - a_i^L)^2
$$
Therefore by the sum rule and chain rule,
$$
    \frac{\partial C}{\partial a_j^L} 
    = \sum_{i=1}^n 2(y_i - a_i^L)(-1)\frac{\partial a_i^L}{\partial a_j^L} 
$$
Assuming the variables $\left(a_1^L,\dots,a_n^L\right)$ are independent, the derivative of any one of them with respect to any other one of them is zero.  But the derivative of each of them with respect to itself is one.  In other words,
$$
    \frac{\partial a_i^L}{\partial a_j^L} = \delta_{ij} = \begin{cases} 1 & i = j \\ 0 & i \neq j \end{cases}
$$
The symbol $\delta_{ij}$ is called the Kronecker delta.
Returning to the summation, we see that each term is multiplied by $0$ except the case $i=j$.  So the only surviving term is that one:
$$
    \frac{\partial C}{\partial a_j^L} = 2(y_j - a_j^L)(-1) = 2(a_j^L-y_j)
$$
A: The notation is more than a little confusing, but I think this is just computing the derivative with respect to a variable that appears in just one of the summands. See what it says with $j=1$ in the answer, leaving the dummy index $j$ in the summation. Or change the dummy $j$ to $i$.
