3
$\begingroup$

Maybe this question is simple, but I really need some help. When we use the Maximum Likelihood Estimation(MLE) to estimate the parameters, why we always put the log() before the joint density? To use the sum in place of product? But why? The wikipedia said it will be convenient. Why? Thank you.

$\endgroup$
  • $\begingroup$ If you get sums instead of many products you get linear functions. Linear functions are easier to compute and to play with. If you are optimizing, it is good to have linear functions. Linear optimization or linear programming has been largely studied. $\endgroup$ – OR. Sep 25 '13 at 1:56
1
$\begingroup$

Some reasons why taking the logarithm of the likelihood can be helpful include:

  1. It turns the likelihood into a sum instead of a product.

  2. In many cases, the individual terms of the sum are logs of exponentials which simplify.

  3. In many cases, the log likelihood is a concave function while the likelihood itself is not concave. When this is the case, convex optimization techniques can be used to find the MLE.

  4. In numerical work, the likelihood is often nonzero but so tiny that it cannot be represented in double precision floating point. For example, suppose that the likelihood is $1.0 \times 10^{-500}$. This is too small to represent in double precision but could easily be the likelihood in a situation where we had 500 data points each with likelihood 0.1. The logarithmic transformation reduces the range of values that have to be dealt with.

$\endgroup$
0
$\begingroup$

by using the log function our likelihood function is become more convenient and finding of its derivative is become more easy that's why we use log function. this is the only reason for using the log. in some cases derivative is not to easily calculate and after taking the log of likelihood function it's become simple. in the problem of MLE our aim is to maximize the likelihood function and after the applying log function there is no change in maximization.

$\endgroup$
0
$\begingroup$

There is one more (in addition to what was pointed out by ABHI and ABC) clear situation when taking log is useful: exponential family of distributions. Taking log helps you to get rid of unpleasant (in root finding process) exponents.

$\endgroup$
0
$\begingroup$

One does not ALWAYS do that. But with an i.i.d. sample, one has $$ L(\theta) = f_\theta(x_1) \cdots f_\theta(x_n) $$ and one seeks the value of $\theta$ that maximizes that. Often (not always) finding the derivative of the function to be maximized is done. Finding $L'(\theta)$ directly is messy because the product rule needs to be applied to a product of many functions. Now look at $$ \log L(\theta) = \log f_\theta(x_1) + \cdots + \log f_\theta(x_n). $$ Finding the derivative can now be done simply by finding the derivative of each term separately and adding them up.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.