Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I was going through this article and they have this log likelihood given by $$ LL = \sum_{i=1}^n A_i\log p_i + \sum_{i=1}^n A'_i\log(1-p_i).$$

Basically this is the loglikelihood of a logistic regression where pi is the output from the sigmoid function and Ai is the number of entries at $i$ having y value 1 and $A'_i$ is the number of entries at $i$ having y value $0$.

Now the close form solution of this is given by

$$p_i = \frac{A_i}{A_i+A'_i}$$

I didn't get this. Where the above solution came from?

share|cite|improve this question
I got the idea from this lecture… – user31820 Jun 25 '12 at 15:59
"the close form solution of this.. " that's actually the solution of the derivative of LL set equal to zero (critical point). – leonbloy Jun 25 '12 at 15:59
Where is the article where you saw this log likelihood? Can you link to it? – Chris Taylor Jun 25 '12 at 16:04
up vote 1 down vote accepted

The derivative of the log-likelihood with respect to $p_i$ is given by $$ \frac{A_i}{p_i}-\frac{A_i'}{1-p_i}. $$ Putting this equal to zero and solving for $p_i$ yields $$ p_i=\frac{A_i}{A_i+A_i'}. $$ Of course you should show that this in fact is a maximum and not a minimum, and this is easily done by looking at the second derivative with respect to $p_i$.

share|cite|improve this answer
And don't forget to check that the function is differentiable in its domain, and that we don't need for check for other (non critical) global maxima (eg, at the borders of the domain) – leonbloy Jun 25 '12 at 17:49

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.