1
$\begingroup$

To calculate the MLE, I see that we can easily take the logarithm of the likelihood function like so: https://en.wikipedia.org/wiki/Exponential_distribution#Maximum_likelihood

I have the following likelihood function:

L(λ) = (λ^3)(e^(-9λ))

I know that I can use the procedure described in the wikipedia article (take the natural log of both sides and find the MLE). Can I also take the derivative and use the product rule?

$\endgroup$
3
$\begingroup$

This has to do with the fact that in calculus, the derivative is a linear operator: $$(f+g)' = f' + g'.$$ But it is not a multiplicative operator: $$(fg)' = f'g + g'f \ne f'g'.$$ So, when we talk about the MLE of a sample, a product naturally arises because the joint distribution of independent observations $(x_1, \ldots, x_n)$ is given by the product of the marginal distributions of each observation; i.e., $$f(x_1, \ldots, x_n \mid \boldsymbol \theta) = \prod f(x_i \mid \boldsymbol \theta).$$ So to find the maximum likelihood, it is usually easier to apply a monotone transformation to the likelihood (thus preserving the location of relative extrema) that converts multiplication to addition--this is the logarithm function.

$\endgroup$
  • $\begingroup$ got it, thanks a lot $\endgroup$ – A user Sep 30 '15 at 1:16
  • 1
    $\begingroup$ would using the product rule work in this case? It seems your answer implies no $\endgroup$ – A user Sep 30 '15 at 1:17
  • 1
    $\begingroup$ In theory, solving for the critical points of $(fg)' = 0$ yields the same values as for solving $(\log f)' + (\log g)' = 0$. But, usually, one way is easier than the other. $\endgroup$ – heropup Sep 30 '15 at 2:01
2
$\begingroup$

Yes. Since the log is a monotonic function, the log-likelihood will have maxima in exactly the same places where the likelihood has maxima. So both methods will give the same answer.

$\endgroup$
1
$\begingroup$

Well, you can use any function that has its maximum at the same place as the likelihood function. This is because you don't care about the value of the function. Instead, using that value, you want to solve for your parameter $\lambda$.

Log function has two nice properties that work very well for us:

  • Its value is the highest wherever the likelihood function is at its highest (the monotonicity that other commenters have already mentioned).
  • Log converts exponents into factors, and products into sums: e.g., $log(x^px^q) = p\ log(x) + q\ log(x)$
  • To compute the maximum (the M in MLE), it is easy to take derivative of a log function, compared to the original likelihood function: $\frac{d}{dx}log(x) = \frac{1}{x}$

There is a nice practical benefit to the second point above, when you're using statistical software. Computers don't have to deal with very small fractions as they keep computing products. Imagine a Bernoulli distribution with $p = 0.2$. For large $n$, $0.2^n$ and $(1-0.2)^n$ quickly become small values. For $n = 10$, the first factor is already $0.0000001024$.

$\endgroup$
  • $\begingroup$ And this was not already fully explained on this page 2.5 years ago? $\endgroup$ – Did Mar 3 '18 at 7:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.