# Understanding partial derivative of logistic regression cost function

I'm following along in Andrew Ng's great lecture series on machine learning, and he presents the following as the cost function for a logistic regression model [link]:

$$L(a,y) = -(y \log(a) + (1 - y) \log(1 - a))$$

He then builds a little math graph, or series of equations, that can be used as helpers for computing the partial derivatives of $L$ with respect to various variables [link]:

$$z = w_1x_1 + w_2x_2 + b$$ $$\hat{y} = a = \sigma(z)$$

Next he says that the following represents the derivative of $L$ wrt $a$ [link]:

$$\frac{\partial L}{\partial a} = -\frac{y}{a} + \frac{1-y}{1-a}$$

Unfortunately, he doesn't give any clues as to how this can be derived. Does anyone here know how to derive this partial derivative given the equations above? I'd be very grateful for any insights others can offer on this question!

• Hmmm... did you realize that the "answer" below does not apply to your setting?
– Did
Commented Dec 31, 2017 at 15:34
• @Did no I didn't--can you help me see why it doesn't apply? Commented Dec 31, 2017 at 15:44
• Because $L(a,y)$ in your question and $L(a,y)$ in this "answer" are not the same.
– Did
Commented Dec 31, 2017 at 15:46
• @Did I don't follow--I take them both to be function arguments but am learning. Can I ask you for more details? Commented Dec 31, 2017 at 15:48
• ?? Simply correct the faulty sign and proceed.
– Did
Commented Dec 31, 2017 at 16:33

$$L(a,y)=-\left(y\log(a)+(1-y)\log(1-a)\right)$$

we get

$$\frac{\partial L(a,y)}{\partial a}=-\left(\frac{y}{a}+\frac{1-y}{1-a}\cdot (-1)\right)$$

which simplifies to

$$-\frac{y}{a} + \frac{1-y}{1-a}$$

since

$$(\log(a))'=\frac{1}{a}$$

and

$$(\log(1-a))'=\frac{1}{1-a}\cdot (-1)$$

using the chain rule

• Thanks very much @Dr.SonhardGraubner this is very helpful. My only remaining doubt is the log(1-a)' term -- could I ask how you applied the chain rule in this case? Commented Dec 30, 2017 at 22:14
• since $$(1-a)'=-1$$ Commented Dec 30, 2017 at 22:15
• is it clear now? Commented Dec 30, 2017 at 22:18
• all the best in the year 2018! Commented Dec 30, 2017 at 22:41
• and if there is a Problem we will solve this Commented Dec 30, 2017 at 22:44