Given the following:

$h=g\left(w_1\cdot x+b_1\right)$

$ z=w_2\cdot h+b_2$

$y_\text{hat} = \mbox{softmax}\left(z\right)$

$\mbox{Loss}\left(y,y_\text{hat}\right)=-\sum \mbox{onehot}\left(y\right)\cdot \log\left(y_\text{hat}\right)$

How can we derivate the loss function: $\frac{\partial L}{\partial z}$?

  • 1
    $\begingroup$ What is $\operatorname{softmax}(z)$ and $onehot$? Make your OP clear by adding the meaning of your notation, otherwise your question will be doomed to be closed and forgotten. $\endgroup$
    – Mittens
    Jan 22, 2022 at 20:46
  • $\begingroup$ Functions have domains and codomains. $\endgroup$ Jan 22, 2022 at 20:56
  • $\begingroup$ What have you yourself attempted? What is the source of this question? $\endgroup$
    – amWhy
    Jan 28, 2022 at 18:14

1 Answer 1


From here softmax, you will see that with your notations $$ \frac{\partial \mathrm{Loss}}{\partial \mathbf{z}} = \mathbf{y}_\mathrm{hat} - \mathrm{onehot}(y) $$ which is particularly simple...


Not the answer you're looking for? Browse other questions tagged .