Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have always had a problem of understanding the big picture of probability and get lost every time I am studying it. I am working on implementing a X-means algorithm for clustering data and the log likelihood function keeps popping up.

Please can anyone give an easy to understand explanation of Likelihood function and the log likelihood function and possible relate to real life examples . Would mind the equation if they can be broken down .

share|cite|improve this question
You might want to explain what mathematical parts of your question are not covered by sections 1 and 2 of this page and whether sections 4 and 5 of the same present the kind of real life examples you are after, or not. – Did Aug 23 '11 at 14:55
up vote 1 down vote accepted

A coin may be biased; you get "heads" with frequency $p\in(0,1)$. The probability of getting two heads in six independent trials is $\binom{6}{2}p^2(1-p)^4$. The likelihood function is $L(p) = \binom{6}{2}p^2(1-p)^4$. It's the probability as a function of $p$ with the "2" held fixed.

With $p$ fixed, the probability of getting "heads" $x$ times, as a function of $x$, is the probability density function (with respect to counting measure, so it's the probability mass function). But with $x$ fixed (in the example above, $x=2$) the same probability density as a function of $p$ (and as a function of $p$ it's not a probability density function) is the likelihood function $$ L(p) = \binom{6}{2}p^2(1-p)^4. $$ The log-likelihood function is merely the logarithm of the likelihood function: $$ \ell(p) = \log\binom{6}{2} + 2\log p + 4\log(1-p). $$ The logarithm is used simply because it's an easier function to differentiate. One does not usually then go on to find $L'(p)$, because usually one just wants the maximum value rather than the rate of change at particular points. $\log$ is an increasing function, so the maximum value of $\ell$ and that of $L$ occur in the same places.

Maximum-likelihood estimation is not the only purpose for which likelihood functions are used. Another purpose is that if one multiplies a prior probability density of $p$ by the likelihood function, and then normalizes, one gets the posterior probability density function of $p$. That is Bayes' theorem. Bayes himself did this originally in the context of the binomial distribution---just as in the example above. In that kind of problem, one generally has no occasion to take the logarithm explicitly.

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.