This example is taken from Geoffrey Hinton's Coursera on neural networks (seen at 13:15).

$$E=\sum_ip_i(t-y_i)^2,\\ \frac{\partial E}{\partial y_i}=p_i(t-y_i)$$

When I calculate this partial derivative I get:


Does the sigma change how we calculate derivatives?


The partial derivative is a linear operator, so we can always pull it inside a finite sum. Assuming your sum is finite, we have \begin{align*} \frac{\partial E}{\partial y_j} &= \frac{\partial }{\partial y_j} \sum_i p_i (t-y_i)^2 \\ &= \sum_i p_i \frac{\partial}{\partial y_j} (t-y_i)^2. \end{align*} Since \begin{equation*} \frac{\partial}{\partial y_j} (t-y_i)^2 = \begin{cases} 0, &\text{ if } i \neq j \\ -2 (t - y_j), &\text{ if } i = j, \end{cases} \end{equation*} we then have \begin{equation*} \frac{\partial E}{\partial y_j} = - 2p_j (t- y_j). \end{equation*} In general, the way I've done it (writing the derivative with a different index than the one in the sum and then pulling it inside the sum and computing the derivative by cases) is probably the best way to handle this. The screenshot you've posed seems incorrect to me, unless there's something I'm missing from the context.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.