I'm reading a book on neural networks, and there's the demonstration for the slope of the error function.

I don't understand the last step, where it differentiates the expression inside the sigmoid function.

Last step and final result

Why is $\frac{\partial}{\partial w_{jk}}\sum_j w_{jk} \cdot o_j$ = $o_j$ ?

All variables are vectors.


2 Answers 2


For fixed index $j_o$ and $k_o$, note that $$ \frac{\text{d}}{\text{d} w_{j_ok_o}} w_{jk} ·o_j = \left\lbrace \begin{array}{rcl} 1\cdot o_{j_o} & \mbox{if} & j_o=j \mbox{ and } k_o=k\\ 0 & \mbox{if} & j_o\neq j \mbox{ or } k_o\neq k \\ \end{array} \right. $$ Then we have \begin{align} \frac{\text{d}}{\text{d} w_{j_ok_o}} %\left\lgroup \sum_j w_{jk} ·o_j %\right\rgroup &= \sum_j \frac{\text{d}}{\text{d} w_{j_ok_o}} w_{jk} ·o_{j} = o_{j_o} \end{align}

  • $\begingroup$ That was very clear, thank you. $\endgroup$ Aug 5, 2017 at 19:18

There is an abuse of notations in your book because $j$ is used as a variable and as an index for the summation.

It would be more correct to write

$$\dfrac{\partial}{\partial w_{jk}} \sum_{j'}w_{j'k}\cdot o_{j'}$$

Among all these $j'$ only one of them is equal to $j$. Therefore the sum is

$$\sum_{j' \neq j}w_{j'k}\cdot o_{j'} + w_{jk}\cdot o_j$$

When you derive this with respect to $w_{jk}$ the derivative of the first sum is $0$.

  • 1
    $\begingroup$ You need to change this to partial derivatives. $\endgroup$
    – adfriedman
    Aug 5, 2017 at 19:02
  • $\begingroup$ I'm sorry, but I don't understand where is j' coming from. I forgot to mention that the variables are all vectors (including w_jk). Edited original question. $\endgroup$ Aug 5, 2017 at 19:15
  • $\begingroup$ @NicodeOry when you write $\sum_j u_j$ it is the same as $\sum_p u_p$ or $\sum_{j'} u_{j'}$. But since $j$ is already used for saying with respect to which vector you want to derive you should not use it to index your sum. $\endgroup$
    – fonfonx
    Aug 5, 2017 at 19:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.