2
$\begingroup$

Here is the expression to take the derivative of. $$C = \frac{1}{2}\sum_j (y_j - a_j^L)^2$$

Here is the result. $$\frac{\partial C}{\partial a_j^L} = 2(a_j^L-y_j)$$

Multiplying by 2, then again by the derivative of the inside (-1) seems reasonable, but what happened to the summation?

$\endgroup$
3
  • 1
    $\begingroup$ All the other variables are constant? $\endgroup$
    – Randall
    Commented Dec 19, 2017 at 19:28
  • $\begingroup$ The "j" and "L" are just indices. This is from: neuralnetworksanddeeplearning.com/chap2.html $\endgroup$
    – compguy24
    Commented Dec 19, 2017 at 19:37
  • $\begingroup$ Right, that’s exactly what I’m saying: You’re taking a partial with respect to only one of them. $\endgroup$
    – Randall
    Commented Dec 19, 2017 at 23:35

3 Answers 3

1
$\begingroup$

Let's save the index $j$ for the derivative and write $$ C = \frac{1}{2} \sum_{i=1}^n (y_i - a_i^L)^2 $$ Therefore by the sum rule and chain rule, $$ \frac{\partial C}{\partial a_j^L} = \sum_{i=1}^n 2(y_i - a_i^L)(-1)\frac{\partial a_i^L}{\partial a_j^L} $$ Assuming the variables $\left(a_1^L,\dots,a_n^L\right)$ are independent, the derivative of any one of them with respect to any other one of them is zero. But the derivative of each of them with respect to itself is one. In other words, $$ \frac{\partial a_i^L}{\partial a_j^L} = \delta_{ij} = \begin{cases} 1 & i = j \\ 0 & i \neq j \end{cases} $$ The symbol $\delta_{ij}$ is called the Kronecker delta.

Returning to the summation, we see that each term is multiplied by $0$ except the case $i=j$. So the only surviving term is that one: $$ \frac{\partial C}{\partial a_j^L} = 2(y_j - a_j^L)(-1) = 2(a_j^L-y_j) $$

$\endgroup$
1
$\begingroup$

The derivative is with respect to a component $a_j^L$ thus the others cancel out.

$\endgroup$
0
$\begingroup$

The notation is more than a little confusing, but I think this is just computing the derivative with respect to a variable that appears in just one of the summands. See what it says with $j=1$ in the answer, leaving the dummy index $j$ in the summation. Or change the dummy $j$ to $i$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .