Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I was going through this article and they have this log likelihood given by $$ LL = \sum_{i=1}^n A_i\log p_i + \sum_{i=1}^n A'_i\log(1-p_i).$$

Basically this is the loglikelihood of a logistic regression where pi is the output from the sigmoid function and Ai is the number of entries at $i$ having y value 1 and $A'_i$ is the number of entries at $i$ having y value $0$.

Now the close form solution of this is given by

$$p_i = \frac{A_i}{A_i+A'_i}$$

I didn't get this. Where the above solution came from?

share|improve this question
    
I got the idea from this lecture ocw.mit.edu/courses/mathematics/… –  user31820 Jun 25 '12 at 15:59
    
"the close form solution of this.. " that's actually the solution of the derivative of LL set equal to zero (critical point). –  leonbloy Jun 25 '12 at 15:59
    
Where is the article where you saw this log likelihood? Can you link to it? –  Chris Taylor Jun 25 '12 at 16:04

1 Answer 1

up vote 1 down vote accepted

The derivative of the log-likelihood with respect to $p_i$ is given by $$ \frac{A_i}{p_i}-\frac{A_i'}{1-p_i}. $$ Putting this equal to zero and solving for $p_i$ yields $$ p_i=\frac{A_i}{A_i+A_i'}. $$ Of course you should show that this in fact is a maximum and not a minimum, and this is easily done by looking at the second derivative with respect to $p_i$.

share|improve this answer
    
And don't forget to check that the function is differentiable in its domain, and that we don't need for check for other (non critical) global maxima (eg, at the borders of the domain) –  leonbloy Jun 25 '12 at 17:49

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.