2
$\begingroup$

I'm following along in Andrew Ng's great lecture series on machine learning, and he presents the following as the cost function for a logistic regression model [link]:

$$L(a,y) = -(y \log(a) + (1 - y) \log(1 - a)) $$

He then builds a little math graph, or series of equations, that can be used as helpers for computing the partial derivatives of $L$ with respect to various variables [link]:

$$ z = w_1x_1 + w_2x_2 + b $$ $$ \hat{y} = a = \sigma(z) $$

Next he says that the following represents the derivative of $L$ wrt $a$ [link]:

$$ \frac{\partial L}{\partial a} = -\frac{y}{a} + \frac{1-y}{1-a} $$

Unfortunately, he doesn't give any clues as to how this can be derived. Does anyone here know how to derive this partial derivative given the equations above? I'd be very grateful for any insights others can offer on this question!

$\endgroup$
  • $\begingroup$ Hmmm... did you realize that the "answer" below does not apply to your setting? $\endgroup$ – Did Dec 31 '17 at 15:34
  • $\begingroup$ @Did no I didn't--can you help me see why it doesn't apply? $\endgroup$ – duhaime Dec 31 '17 at 15:44
  • $\begingroup$ Because $L(a,y)$ in your question and $L(a,y)$ in this "answer" are not the same. $\endgroup$ – Did Dec 31 '17 at 15:46
  • $\begingroup$ @Did I don't follow--I take them both to be function arguments but am learning. Can I ask you for more details? $\endgroup$ – duhaime Dec 31 '17 at 15:48
  • 1
    $\begingroup$ ?? Simply correct the faulty sign and proceed. $\endgroup$ – Did Dec 31 '17 at 16:33
1
$\begingroup$

if your equation is

$$L(a,y)=-\left(y\log(a)+(1-y)\log(1-a)\right)$$

we get

$$\frac{\partial L(a,y)}{\partial a}=-\left(\frac{y}{a}+\frac{1-y}{1-a}\cdot (-1)\right)$$

which simplifies to

$$-\frac{y}{a} + \frac{1-y}{1-a}$$

since

$$(\log(a))'=\frac{1}{a}$$

and

$$(\log(1-a))'=\frac{1}{1-a}\cdot (-1)$$

using the chain rule

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks very much @Dr.SonhardGraubner this is very helpful. My only remaining doubt is the log(1-a)' term -- could I ask how you applied the chain rule in this case? $\endgroup$ – duhaime Dec 30 '17 at 22:14
  • $\begingroup$ since $$(1-a)'=-1$$ $\endgroup$ – Dr. Sonnhard Graubner Dec 30 '17 at 22:15
  • $\begingroup$ is it clear now? $\endgroup$ – Dr. Sonnhard Graubner Dec 30 '17 at 22:18
  • 1
    $\begingroup$ all the best in the year 2018! $\endgroup$ – Dr. Sonnhard Graubner Dec 30 '17 at 22:41
  • 1
    $\begingroup$ and if there is a Problem we will solve this $\endgroup$ – Dr. Sonnhard Graubner Dec 30 '17 at 22:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.