I'm currently reading the book Deep Learning (Goodfellow et al., 2015) and had a question regarding the calculation of a gradient when explaining backpropagation for a certain example. For anyone who's curious, this is from section 6.5.9: Differentiation outside the Deep Learning Community.

Suppose we have variables $p_1, p_2, ... , p_n$ representing probabilities and variables $z_1, z_2, ... , z_n$ representing unnormalized log probabilities. Suppose we define

$$q_i = \frac{e^{z_i}}{\sum_i e^{z_i}}$$

where we build the softmax function out of exponentiation, summation and division operations, and construct a cross-entropy loss $J = -\sum_i p_i \log{q_i}$. A human mathematician can observe that the derivateive of $J$ with respect to $z_i$ takes a very simple form: $q_i - p_i$.

I don't know how this result was derived, and was hoping that someone could give me some tips or advice. What I have so far is

$$\log{q_i} = \log{e^{z_i}} - \log({\sum_i e^{z_i}})$$

$$ \begin{align} p_i\log{q_i} & = p_i \log{e^{z_i}} - p_i \log({\sum_i e^{z_i}}) \\ & = p_iz_i - p_i\log(\sum_i e^{z_i}) \end{align}$$

If we take the derivative of $J = p_i\log{q_i}$ then I can understand that $d/dz_i (p_i z_i) = p_i$, but how do we differentiate the second term that contains the logarithm of the summation?

Thank you.


Your derivation of $p_i\log q_i$ is fine. Based upon it we obtain for $J$:

\begin{align*} J&=-\sum_{j=1}^np_jz_j+\sum_{j=1}^np_j\log\left(\sum_{k=1}^ne^{z_k}\right)\\ &=-\sum_{j=1}^np_jz_j+\log\left(\sum_{k=1}^ne^{z_k}\right)\tag{1} \end{align*}

In the last line we use the sum of the probabilities $p_j,1\leq j\leq n$ is equal to $1$.

From (1) we obtain the derivation of $J$ with respect to $z_i$ as: \begin{align*} \color{blue}{\frac{d}{dz_i}J} &=\frac{d}{dz_i}\left(-\sum_{j=1}^np_jz_j\right)+\frac{d}{dz_i}\left(\log\left(\sum_{k=1}^ne^{z_k}\right)\right)\\ &=-p_i+\frac{e^{z_i}}{\sum_{k=1}^ne^{z_k}}\\ &\,\,\color{blue}{=-p_i+q_i} \end{align*} in accordance with the claim.


Well, $$\frac{\partial J}{\partial z_i} = \frac{\partial}{\partial z_i} (-\sum_j p_j\log q_j )=\frac{\partial}{\partial z_i}( -p_i\log q_i) \\= \frac{\partial}{\partial z_i} (-p_i \log (e^{z_i}/\sum_k e^{z_k}))\\ =\frac{\partial}{\partial z_i} (-p_iz_i\log e + p_i\log(\sum_k e^{z_k})) \\= \frac{\partial}{\partial z_i} (-p_iz_i) + \frac{\partial}{\partial z_i}p_i\log(\sum_k e^{z_k}) \\= -p_i + p_ie^{z_i}/\sum_k e^{z_k} \\= -p_i+p_iq_i.$$

  • $\begingroup$ But didn't the op claim the partial derivative to be $q_i - p_i$? $\endgroup$ – Viktor Glombik Jul 19 at 9:00
  • $\begingroup$ Needs checking by a human mathematician. $\endgroup$ – Wuestenfux Jul 19 at 9:07
  • $\begingroup$ Hello, thanks for the answer. In the first line why (or how) did you get rid of the symbol for summation? Is this a rule in calculus? $\endgroup$ – Seankala Jul 20 at 1:42
  • $\begingroup$ @Seankala: Note $p_1$ is a function of $z_1,$ but $p_2$ is not related to $z_1.$ $p_i$ for $i \neq 1 $ are all constant with respect to $z_1.$ The same argument works for the $q_i.$ The derivative is linear, and the derivative of a constant is $0.$ $\endgroup$ – Artimis Fowl Jul 21 at 21:49
  • 1
    $\begingroup$ @Wuestenfux I think there is a mistake in the third to last line - note $z_i = \log p_i \implies p_i = e^{z_i},$ so we can't treat $p_i$ as a constant there. Likewise for $q_i.$ $\endgroup$ – Artimis Fowl Jul 21 at 21:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.