I include a bit of an introduction, even though my main question is more mathematical. I was tasked with finding the Maximum Likelihood Estimate for $\theta$ in $$\mathrm P(X>x) = \left(\frac ax \right)^\theta $$ where $X$ is a variable, and $x$ represents a value that variable can take on. The Probability Density Function is $\newcommand\diff[2]{\frac{\mathrm d#1}{\mathrm d#2}}\diff Fx=\frac{-\theta a^\theta}{x^{\theta + 1}}$, where $F = \mathrm P(X>x)$. I maximise the loglikelihood function $l = \ln(-\theta) + \theta \ln a - (\theta + 1)\ln x\ $ to get $\hat\theta(x_i) = \frac 1{\ln x_i - \ln a}$, where the $\hat.$ indicates that $\hat\theta$ is an estimate of $\theta$, based on the data sample. Now, the answer is supposed to be $$\hat\theta = \frac 1{\overline {\ln x} - \ln a}$$ where $\overline {\phantom{x}}$ indicates the average: $\overline{\ln x} = \frac 1n \sum_i \ln x_i$. I am stumped as to how to get this answer directly from $\hat\theta(x_i)$.

Does $$\frac 1n \sum_i \frac 1{\ln x_i - \ln a} = \frac 1{\overline {\ln x} - \ln a}\qquad ?$$

I think $\frac 1n \sum_i \widehat{\frac 1{\theta(x_i)}} = \frac 1n\sum_i (\ln x_i - \ln a) =\overline{\ln x} - \ln a = \widehat {\frac 1\theta} \implies \hat\theta = \frac 1{\overline{\ln x} - \ln a}$, but is this the only way to show the above?

  • $\begingroup$ Several errors in the post. The probability density function is the derivative of the distribution function $P(X\le x)$. And please mention the support/domain of the density, where it is defined. This is important to derive the maximum likelihood estimator. $\endgroup$ – StubbornAtom Jan 29 at 16:24
  • 1
    $\begingroup$ Looks like $a$ is known here; this should be mentioned. $\endgroup$ – StubbornAtom Jan 29 at 16:45
  • $\begingroup$ @StubbornAtom yes, I can see now that my PDF was calculated incorrectly. I think I included the introduction to check if I got any of it wrong. $a$ is "known". See the original document here (only a few cut pages). $\endgroup$ – ahorn Jan 30 at 6:31
  • $\begingroup$ @ahorn Is there still a question which has to be clarified? $\endgroup$ – callculus Jan 30 at 20:53

Usually the maximum likelihood estimator is calculated based on the pdf of $X$.

If $P(X> x)=\left( \frac{\alpha}{x}\right)^{\theta}$ then $P(X\leq x)=1-\left( \frac{\alpha}{x}\right)^{\theta}$

Differentating w.r.t. $\theta$

$f_X(x)=\frac{\theta a^\theta}{x^{\theta + 1}}$

Then the likelihood function is

$L(\theta)=\Pi_{i=1}^n \frac{\theta a^\theta}{x_i^{\theta + 1}}$

$L(\theta)=\theta^n a^{\theta\cdot n}\cdot \Pi_{i=1}^n x_i^{-1-\theta}$

Taking ln

$$\ln L=n\cdot \ln (\theta)+n\cdot \theta\cdot \ln (\alpha)+\sum\limits_{i=1}^{n} (-1-\theta)\cdot \ln (x_i)$$

I think you can differentiate $\ln L$, set it equal to $0$ and solve the equation for $\theta$.

The result is indeed $\hat\theta = \frac 1{\overline {\ln x} - \ln a}$, where $\overline {\ln x}=\frac{ 1}{n}\sum\limits_{i=1}^{n} \ln (x_i)$

  • $\begingroup$ I'm just stuck on the part where you state what the likelihood function is. I haven't dealt with likelihood functions much (and I just assumed it would be the PDF), so perhaps I could find out more on Wikipedia? $\endgroup$ – ahorn Jan 31 at 3:57
  • $\begingroup$ @ahorn Sure you can make a search at the net or just here at MSE. The key word is "Maximum likelihood estimation (MLE)". In my opinion this site is more comprehensible than Wiki. Basically you have an observation of $n$ data. They are fix. Then you multiply n-times the probability density function (pdf) where the observed data $x_i$ are given. This is the likelihood function. In your case the parameter $\alpha$ is given as well. Now you are looking for the maximum of the likelihood function, $\endgroup$ – callculus Jan 31 at 6:07
  • $\begingroup$ @ahorn (Continued): In many cases it can be done by differentiating the likelihood function or even by differentiating the logarithm of the function. But the MLE cannot always found by differentiation. An example is the uniform distribution. $\endgroup$ – callculus Jan 31 at 6:11
  • $\begingroup$ I think I can think of the likelihood function as evaluating the n-dimensional PDF with domain $\prod_{i=1}^nX_i$ at the point $\mathbf x = (x_1, x_2, \dots , x_n)$, then maximizing that value w.r.t. $\theta$. $\endgroup$ – ahorn Jan 31 at 8:29
  • $\begingroup$ @ahorn It sounds like a useful interpretation. $\endgroup$ – callculus Jan 31 at 9:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.