Let $X_1, X_2,\ldots$ be a random sample from the probability density function for $0<\mu<\infty$, $0<\alpha < 1$

$$f(x;\mu,\alpha)= \begin{cases}\frac{1}{\Gamma(\alpha)}(x-\mu)^{\alpha-1}e^{-(x-\mu)}, & x>\mu \\ 0, & \text{otherwise} \end{cases}$$

Does Maximum Likelihood Estimator exist for parameters $\alpha$ and $\mu$ ?

By observation, this looks like a Weibull distribution. Also, I know that we can obtain MLE by taking product of the PDF from $1$ to $n$ and differentiating wrt to the parameter.

But I do not know about the criteria that a parameter needs to satisfy in order to have Maximum Likelihood Estimator.

  • $\begingroup$ It's a shifted Gamma distribution with a known scale parameter. This problem differs from that of estimating the parameters of a Gamma distribution in that one must estimate the shift. $\endgroup$ – Michael Hardy Oct 8 '17 at 16:21
  • $\begingroup$ $$L(\mu,\alpha) = \frac 1 {\Gamma(\alpha)^n} \left( \prod_{i=1}^n (x_i-\mu) \right)^{\alpha-1} \exp \left( -\sum_{i=1}^n (x_i-\mu) \right) \quad \text{for } \mu \le \min\{x_1,\ldots,x_n\} \text{ and } \alpha>0.$$ $$\ell(\mu,\alpha) = \log L(\mu,\alpha) = -n\log\Gamma(\alpha) + (\alpha-1) \sum_{i=1}^n \log(x_i-\mu) - \sum_{i=1}^n (x_i-\mu) \\ \text{for } \mu\le\min \text{ and } \alpha>0. $$ $\endgroup$ – Michael Hardy Oct 9 '17 at 0:57
  • $\begingroup$ MLE of $\alpha$ can be found numerically. $\endgroup$ – StubbornAtom Oct 9 at 6:38

\begin{align} & \text{For } \mu \le \min\{x_1,\ldots,x_n\} \text{ and } \alpha>0, \text{we have} \\[10pt] L(\mu,\alpha) & = \frac 1 {\Gamma(\alpha)^n} \left( \prod_{i=1}^n (x_i-\mu) \right)^{\alpha-1} \!\!\! \exp \left( -\sum_{i=1}^n (x_i-\mu) \right), \\[10pt] \ell(\mu,\alpha) & = \log L(\mu,\alpha) = -n\log\Gamma(\alpha) + (\alpha-1) \sum_{i=1}^n \log(x_i-\mu) - \sum_{i=1}^n (x_i-\mu). \end{align}

You gave us $\alpha<1.$

That implies $\alpha-1<0,$ so that $\ell(\mu,\alpha)$ is an increasing function of $\mu$ until $\mu$ gets as big as $\min\{x_1,\ldots,x_n\}.$

Therefore $\widehat\mu = \min\{x_1,\ldots,x_n\}.$ If we didn't have the constraint that $\alpha<1,$ then this would be more complicated.

This value of $\widehat\mu$ does not depend on $\alpha$ as long as $\alpha$ remains in that interval. Therefore we can just plug in $\min$ for $\mu$ and then seek the value of $\alpha\in(0,1)$ that maximizes $\ell(\min,\alpha).$

Now we have $$ \ell(\min,\alpha) = -n\log\Gamma(\alpha) + (\alpha-1)A + \big( \text{constant} \big) $$ where "constant" means not depending on $\alpha.$ $$ \frac {\partial\ell}{\partial\alpha} = -n\frac{\Gamma'(\alpha)}{\Gamma(\alpha)} + A. $$ Etc.

  • $\begingroup$ So neither $\alpha$ nor $\mu$ have MLE right? $\endgroup$ – The Doctor Oct 9 '17 at 3:32
  • $\begingroup$ $\mu$ does have an MLE, and I said what it is. $\endgroup$ – Michael Hardy Oct 9 '17 at 4:06
  • 1
    $\begingroup$ I think the question of the MLE of $\alpha$ is somewhat more involved. If $A<0,$ then I suspect there is an MLE strictly between $0$ and $1. \qquad$ $\endgroup$ – Michael Hardy Oct 9 '17 at 4:10

The maximum likelihood estimator may not exist if the likelihood function is not concave and the parameter space is unbounded. In such a situation, the maximum may be attained at the asymptote (e.g., $\infty$).

As a toy example, suppose that $X \sim N(\mu, \sigma^2)$. Then, the log likelihood if we observe $X = \mu$ reduces to $LL \propto -\log(\sigma)$. Maximizing the above likelihood requires that $\sigma \rightarrow 0$.

  • $\begingroup$ Can you please explain in a bit more detail? $\endgroup$ – The Doctor Oct 8 '17 at 15:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.