I am following Andrew NG's Machine Learning course on Coursera.

The cost function without regularization used in the Neural network course is:

$J(\theta) = \frac{1}{m} \sum ^{m}_{i=1}\sum ^{K}_{k=1} [-y_{k}^{(i)}log((h_{\theta}(x^{(i)}))_{k}) -(1-y_{k}^{(i)})log(1-(h_{\theta}(x^{(i)}))_{k})]$

, where $m$ is the number of examples, $K$ is the number of classes, $J(\theta)$ is the cost function, $x^{(i)}$ is the i-th training example, $\theta$ are the weight matrices and $h_{\theta}(x^{(i)})$ is the prediction of the neural network for the i'th training example.

I understand intuitively that the backpropagation error associated with the last layer(h) is h-y. Nevertheless, I want to be able to prove this formally.

For simplicity, I considered m = K = 1:

$J(\theta) = -y \log(h_{\theta}) - (1-y) \log(1-h_{\theta})$

and tried to prove this to myself on paper but wasn't able to.

Neural Network Definition:

This neural network has 3 layers. (1 input, 1 hidden, 1 output).

It uses the sigmoid activation function,

$\sigma(z) = \frac{1}{1+e^{-z}}$. The input is $x$.

Input layer: $a^{(1)} = x$. (add bias $a_{0}^{(1)}$).

Hidden Layer: $z^{(2)} = \Theta^{(1)}a^{(1)}$ , $a^{2} = \sigma(z^{(2)})$, (add bias $a_{0}^{(2)}$).

Output layer: $z^{(3)} = \Theta^{(2)}a^{(2)}$ , $a^{3} = \sigma(z^{(3)}) = h_{\theta}(x)$.

During backpropagation, $\delta^{(3)}$ is the error associated with the output layer.


  1. Why is it that:

$\delta^{(3)} = h_{\theta} - y$ ?

  1. Shouldn't:

$\delta^{(3)} = \frac{\partial {J}} {\partial {h_{\theta}}}$ ?

  • 1
    $\begingroup$ The notations are horrible. Let $z^{(i)} = h_\theta(x^{(i)})$ the output for the $i$th input $x^{(i)}$. What we want (to apply the gradient descent) is $\frac{\partial J}{\partial \theta_{lj}}$, and for this we look at $\frac{\partial J}{\partial z^{(i)}_j}$ and $\frac{\partial z^{(i)}_j}{\theta_{lj}}$. Also you didn't define your neural network (ie. $h_\theta$) only the objective function. $\endgroup$ – reuns Jun 15 '17 at 16:07
  • 1
    $\begingroup$ You meant $$h_\theta(1-h_\theta) \frac{\partial J}{\partial h_{\theta}} = h_{\theta}-y$$ $\endgroup$ – reuns Jun 15 '17 at 16:18
  • 1
    $\begingroup$ This question is missing context or other details: Please improve the question by providing additional context, which ideally includes your thoughts on the problem and any attempts you have made to solve it. This information helps others identify where you have difficulties and helps them write answers appropriate to your experience level. $\endgroup$ – The Great Duck Jun 15 '17 at 16:43
  • 1
    $\begingroup$ Related: link $\endgroup$ – user3658307 Jun 15 '17 at 17:01
  • 1
    $\begingroup$ @Roland why are pinging me? That is an auto-generated post because i have flagged your question. You're question lacks context and that is for you to fix. Don't ask me to fix your "homework" question. $\endgroup$ – The Great Duck Jun 15 '17 at 17:05

First, since your cost function is using the binary cross-entropy error $\mathcal{H}$ with a sigmoid activation $\sigma$, you can see that: \begin{align} \frac{\partial J}{\partial h_\theta} &= \frac{1}{m}\sum_i\sum_k\frac{\partial }{\partial h_\theta}\mathcal{H}\left(y_k^{(i)},h_\theta(x^{(i)})_k\right) \\ &= \frac{1}{m}\sum_i\sum_k \left[ \frac{-y_k^{(i)}}{h_\theta(x^{(i)})_k} + \frac{1-y_k^{(i)}}{1-h_\theta(x^{(i)})_k} \right] \\ &= \frac{1}{m}\sum_i\sum_k \frac{h_\theta(x^{(i)})_k - y_k^{(i)}}{ h_\theta(x^{(i)})_k(1-h_\theta(x^{(i)})_k) } \end{align} Hence, for $m=K=1$, as a commenter notes $$ \frac{\partial J}{\partial h_\theta} = \frac{h_\theta - y}{ h_\theta(1-h_\theta) } $$ But this is not so useful, as it computes how the error changes as the final output changes. What you really want is how the cost changes as the weights $\theta^{(\ell)}_{ij}$ are varied, so you can do gradient descent on them. An intermediate calculation is to compute the variation with respect to the activation $ h_\theta=\sigma(z)$. Let the last layer be $s$. Then the output layer error is: \begin{align} \delta^{(s)}_j &= \frac{\partial J}{\partial z_j^{(s)}}\\ &= \frac{1}{m}\sum_i\sum_k \frac{\partial }{\partial z_j^{(s)}} \mathcal{H}\left(y_k^{(i)},h_\theta(x^{(i)})_k\right) \\ &= \frac{-1}{m}\sum_i\sum_k y_k^{(i)} \frac{1}{h_\theta(x^{(i)})_k}\frac{\partial h_\theta(x^{(i)})_k}{\partial z_j^{(s)}} + (1-y_k^{(i)})\frac{1}{1-h_\theta(x^{(i)})_k}\frac{\partial h_\theta(x^{(i)})_k}{\partial z_j^{(s)}} \\ &= \frac{-1}{m}\sum_i\sum_k [1-h_\theta(x^{(i)})_k]y_k^{(i)} - h_\theta(x^{(i)})_k[1-y_k^{(i)}]\\ &= \frac{1}{m}\sum_i\sum_k h_\theta(x^{(i)})_k -y_k^{(i)} \end{align} using the fact that $$ \frac{\partial h_\theta(x^{(i)})_k}{\partial z_j^{(s)}} = \sigma'(z_j^{(s)}) = \sigma(z_j^{(s)})[1-\sigma(z_j^{(s)})] = h_\theta(x^{(i)})_k[1-h_\theta(x^{(i)})_k] $$ So in the case that $m=K=1$ and $s=3$, we have: $$ \delta^{(3)} = h_\theta - y $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.