In the book "Deep Learning" by Ian Goodfellow, Yoshua Bengio and Aaron Courville there is a derivation of the derivative of a loss function that has a very simple form, but is apparently tricky to derive automatically. I have tried to derive the simplified form myself, but could not arrive at the same outcome given in the book.

"For example, suppose we have variables $p_1 , p_2 , ... , p_n$ representing probabilities and variables $z_1 , z_2 , . . . , z_n$ representing unnormalized log probabilities. Suppose we define $$q_i = \frac{exp(z_i)}{\sum_i exp(z_i)} \ \ \ \ \ \ \ \ \ \ (6.5.7)$$,

where we build the softmax function out of exponentiation, summation and division operations, mathematician and can construct observe that a cross-entropy the derivative loss of $J = - \sum_i p_i \ log \ q_i$. A human mathematician can observe that the derivative of J with respect to $z_i$ takes a very simple form: $q_i − p_i$. The back-propagation algorithm is not capable of simplifying the gradient this way, and will instead explicitly propagate gradients through all of the logarithm and exponentiation operations in the original graph." (CHAPTER 6. DEEP FEEDFORWARD NETWORKS, page 215)

So basically we want to find the partial derivative of J with respect to $z_i$ $$\frac{\partial}{\partial z_i} J = $$ $$\frac{\partial}{\partial z_i} - \sum_i p_i \ log \ q_i $$

With respect to $q_i$ I note that the value of $z_i$ is a specific value while the denominator goes over all values of $z_i$. Hence, in my opinion it is clearer to write:

$$q_i = \frac{exp(z_i)}{\sum_j exp(z_j)} $$,

Using this notation I get:

$$\frac{\partial}{\partial z_i} J = \frac{\partial}{\partial z_i} -\sum_i p_i \ log \frac{exp(z_i)}{\sum_j exp(z_j)} $$

One thing that is still unclear in this notation is that $z_i$ has presumably nothing to do with the indices of $p_i$ in the first summation, so again I'd rather write this as:

$$\frac{\partial}{\partial z_i} J = \frac{\partial}{\partial z_i} -\sum_k p_k \ log \frac{exp(z_i)}{\sum_j exp(z_j)} $$

Now in this form I start the computing this derivative.

I notice that we can use the rule $log \frac{a}{b} = log(a) - log(b)$

To rewrite:

$$\frac{\partial}{\partial z_i} -\sum_k p_k \ (log(exp(z_i)) - log({\sum_j exp(z_j)})) = $$

$$\frac{\partial}{\partial z_i} -\sum_k p_k \ (z_i - log({\sum_j exp(z_j)}))$$

Next I note, using the chain rule that:

$$\frac{\partial}{\partial z_i}log({\sum_j exp(z_j)} = \frac{\partial}{\partial z_i}log(g(z_i))$$ $$\frac{1}{\sum_j exp(z_j)} \times \frac{\partial}{\partial z_i}({\sum_j exp(z_j)}) = $$ $$ \frac{1}{\sum_j exp(z_j)} \times exp(z_i)) = $$ $$ \frac{exp(z_i)}{\sum_j exp(z_j)} $$

Using this result, we get also: $$\frac{\partial}{\partial z_i} z_i - log({\sum_j exp(z_j)}) = $$ $$ 1 - \frac{exp(z_i)}{\sum_j exp(z_j)} $$

Finally we need the product rule for differentiation: $\frac{\partial}{\partial x} f(x) * g(x) = f'(x) g(x) + f(x) g'(x)$ This is used to compute the combined derivative over the whole function J, noting that the derivative of every $p_i$ with respect to $z_i$ is always zero.

Hence the combined derivative becomes:

$$\frac{\partial}{\partial z_i} J = \frac{\partial}{\partial z_i} -\sum_k p_k \ log \frac{exp(z_i)}{\sum_j exp(z_j)} = $$ $$-\sum_k p_k (1 - \frac{exp(z_i)}{\sum_j exp(z_j)} = $$ (substituting back the formula of $q_i$) $$-\sum_i p_i (1 - q_i) $$

Finally we can use the fact, that (presumably, this is not explicitly stated but seemingly assumed!) $\sum_k p_k = 1$. We infer this from the fact that $p_1, .., P_n$ are probabilities, presumably it is implied they sum to 1.

So using this we get, $$-\sum_i p_i (1 - q_i) = \bf{q_i - 1}$$

So I arrive at the final simplified form $$\bf{q_i - 1}$$ as opposed to $$\bf{q_i - p_i}$$ given in the book.

My question is: could someone please verify my derivation, is there a mistake in it, or could it be that my outcome is correct and there is an error in the final result of the derivation in the book?


1 Answer 1


Equation (6.5.7) is already somewhere between misleading and wrong: the variable $i$ is bound twice, once as an index of summation and once in the definition of $q_i$. Better write $q_i=\frac{\exp(z_i)}{\sum_j\exp(z_j)}$. Similarly, to avoid confusion, we should derive $J$ with respect to $z_k$. Then we get $$ \frac{\partial}{\partial z_k} J = -\sum_i p_i \frac{\partial}{\partial z_k}\log\frac{\exp(z_i)}{\sum_j\exp(z_j)} = -\sum_i p_i \frac{\partial}{\partial z_k}\log\exp(z_i) + \sum_i p_i \frac{\partial}{\partial z_k}\log\left(\sum_j\exp(z_j)\right). $$ In the first sum only the term $i=k$ depends on $z_k$, which yields $p_k$. In the derivative in the second sum, the index $i$ does not appear, so we can pull out the derivative, and as $\sum p_i=1$ we get $$ \frac{\partial}{\partial z_k} J = -p_k + \frac{\partial}{\partial z_k}\log\left(\sum_j\exp(z_j)\right) = -p_k + \frac{\frac{\partial}{\partial z_k}\sum_j\exp(z_j)}{\left(\sum_j\exp(z_j)\right)} = -p_k + \frac{\exp(z_k)}{\left(\sum_j\exp(z_j)\right)} = -p_k+q_k. $$

  • $\begingroup$ Thanks a lot for the clear answer Jan-Christoph. I realize now that I had made a mistake while splitting the equation and computing the derivatives for the two parts. Your answer shows nicely how in the derivative of the first part only p_k remains while in the derivative of the second part indeed the sum of probabilities up to one is exploited. $\endgroup$
    – GEW
    Commented Sep 11, 2017 at 13:08
  • $\begingroup$ And you are right, the crucial part is to couple p_i in the left of the equation with exp(z_i) in the enumerator; only re-indexing the denominator to a sum over j as you did. That is mainly were I went wrong. $\endgroup$
    – GEW
    Commented Sep 11, 2017 at 13:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .