4
$\begingroup$

Why do we use natural log for MLE?

$\endgroup$
5
$\begingroup$

Likelihood function is generally ratio of two densities. Since working with ratios is not convenient when taking max, one normally takes the logs. This is most effective when densities involve exponential functions and is extremely convenient in the case of normal (Gaussian) densities.

Taking logs does not always help. In fact if the underlying densities are uniform, then logs actually make it worse, not better.

Added based on comments

Say you want to take the derivative of ratio $N/D$ to find the maximum. Then the derivative would be $\frac{N'D - N D'}{D^2}$. This could be difficult to manage. Suppose $L_N = \log(N)$ and $L_D = \log(D)$. Then taking the log of the ratio we get $$ \log\left(\frac{N}{D}\right) = \log(N)-\log(D) = L_N - L_D $$ maximizing a positive functio is same as maximizing the log. So $$ \log\left(\frac{N}{D}\right)' = = L_N'- L_D' \tag1 $$ If $L_N$ and $L_D$ are simple, then the above derivative is simpler to work with.

Note one can obtain the same anser by looking at $$ \frac{D}{N} \left(\frac{N}{D}\right)' \tag2 $$ called the Logarithmic derivative and is obtained from (1). It is not difficult to show that (1) and (2) give the same right hand side.

$\endgroup$
  • $\begingroup$ I try to understand what you have told me but it is still far too abstract. I feel that I kind of understand and at the same time I feel that I am not really understanding what you are telling me... hope you can explain it with a numerical example. Thanks in advance. $\endgroup$ – user122358 Feb 21 '14 at 3:25
2
$\begingroup$

The fundamental reason for the presence of log, the Kullback-Liebler divergence. That is why MLE is consistent, among other things. Pointing out trivial consequences like "log turns product into sum..." is very misleading.

$\endgroup$
1
$\begingroup$

Since natural log is a strictly increasing function, the max of the density in question will be the same as the max of the natural log transformation, given that it exists. The natural log simplifies densities that involve exponentials. Also since densities usually involve products, the transformation will simplify all that potentially messy calculation.

$\endgroup$
0
$\begingroup$

Sometimes we don't use the log (natural or otherwise).

Mostly it's a matter of convenience - taking logs in many cases makes it simpler to find the argmax. However, results like Wilks' theorem may make it more convenient to work with logs in more situations than might otherwise be apparent.

Taking logs is not so much help with the triangular distribution, for example.

$\endgroup$
0
$\begingroup$

I would disagree with Michael's statement that "Pointing out trivial consequences like "log turns product into sum..." is very misleading."

In fact, those consequences are precisely WHY the logs are used when finding the MLE for a parameter. The MLE has the value that it has in order to minimize the Kullback-Liebler divergence. However, I think this question was more about why we USE logs to find the value of the MLE. The answer to that question is what most of the above answers are suggesting - most of the time, the utilization of logarithms (which fortunately do not affect the logic involved in the process) make the calculus and resulting algebra easier. They are a convenience in this setting, not a necessity.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.