# Derivative of sum

I'm reading a book on neural networks, and there's the demonstration for the slope of the error function.

I don't understand the last step, where it differentiates the expression inside the sigmoid function.

Last step and final result

Why is $\frac{\partial}{\partial w_{jk}}\sum_j w_{jk} \cdot o_j$ = $o_j$ ?

All variables are vectors.

For fixed index $j_o$ and $k_o$, note that $$\frac{\text{d}}{\text{d} w_{j_ok_o}} w_{jk} ·o_j = \left\lbrace \begin{array}{rcl} 1\cdot o_{j_o} & \mbox{if} & j_o=j \mbox{ and } k_o=k\\ 0 & \mbox{if} & j_o\neq j \mbox{ or } k_o\neq k \\ \end{array} \right.$$ Then we have \begin{align} \frac{\text{d}}{\text{d} w_{j_ok_o}} %\left\lgroup \sum_j w_{jk} ·o_j %\right\rgroup &= \sum_j \frac{\text{d}}{\text{d} w_{j_ok_o}} w_{jk} ·o_{j} = o_{j_o} \end{align}

• That was very clear, thank you. Aug 5, 2017 at 19:18

There is an abuse of notations in your book because $j$ is used as a variable and as an index for the summation.

It would be more correct to write

$$\dfrac{\partial}{\partial w_{jk}} \sum_{j'}w_{j'k}\cdot o_{j'}$$

Among all these $j'$ only one of them is equal to $j$. Therefore the sum is

$$\sum_{j' \neq j}w_{j'k}\cdot o_{j'} + w_{jk}\cdot o_j$$

When you derive this with respect to $w_{jk}$ the derivative of the first sum is $0$.

• You need to change this to partial derivatives. Aug 5, 2017 at 19:02
• I'm sorry, but I don't understand where is j' coming from. I forgot to mention that the variables are all vectors (including w_jk). Edited original question. Aug 5, 2017 at 19:15
• @NicodeOry when you write $\sum_j u_j$ it is the same as $\sum_p u_p$ or $\sum_{j'} u_{j'}$. But since $j$ is already used for saying with respect to which vector you want to derive you should not use it to index your sum. Aug 5, 2017 at 19:17