2
$\begingroup$

$$\mathbf{X} = \{X_1,X_2,\dots,X_n\}$$ sequence of i.i.d. RV's. Let the distribution of the RV's be defined by $$f(x|\theta)=\frac{\theta}{x^{\theta+1}}, \quad x>1, \quad \theta>1$$ I am interested in the asymptotic variance of the MLE for $\theta$. That is; $$\lim_{n\rightarrow \infty}\mathbb{V}ar_{\theta^*}(\hat{\theta}_n) = \lim_{n\rightarrow \infty}\mathbb{V}ar_{\theta^*}\left( \frac{n}{\sum_{i=1}^n X_i}\right)$$

What I thought is to use the delta-method (1-st order) approximation: $$\mathbb{V}ar_{\theta^*}(\hat{\theta}_n(\mathbf{X})) \approx \left(\hat{\theta}_n\left(\mathbb{E}_{\theta^*}[\mathbf{X}]\right) \right)^2\mathbb{V}ar_{\theta^*}(\mathbf{X})$$

In the above, subscript $\theta^*$ means with respect to the density defined by the true parameter value $\theta^*$ (assuming convergence in probability). Also, the expected value of one RV equals:

$$\mathbb{E}_{\theta^*}[X]=\frac{\theta}{1+\theta}$$ If I'm correct. Would the calculation of the asymptotic variance be possible using the delta-method approximation? I could not get far at all.

$\endgroup$
1
  • $\begingroup$ Why do you think the variance exists? $\endgroup$
    – Did
    Apr 21, 2015 at 6:47

1 Answer 1

1
$\begingroup$

Your MLE for $\theta$ does not seem correct. If $\boldsymbol x = (x_1, \ldots, x_n)$ is an iid sample drawn from a $X \sim \operatorname{Pareto}(1,\theta)$ distribution with density $$f_X(x) = \theta x^{-\theta-1} \mathbb{1}(x > 1)$$ then the likelihood function is $$\mathcal L (\theta \mid \boldsymbol x) = \theta^n \left(\prod_{i=1}^n x_i \right)^{-\theta-1} \mathbb{1}(x_{(1)} > 1).$$ The log-likelihood is then $$\ell(\theta \mid \boldsymbol x) = n \log \theta - (\theta+1) \sum_{i=1}^n \log x_i + \log \mathbb{1}(x_{(1)} > 1),$$ which suggests letting $\overline{ \log x} = \frac{1}{n} \sum_{i=1}^n \log x_i$. Solving for the critical points gives $$0 = \frac{\partial \ell}{\partial \theta} = \frac{n}{\theta} - n \, \overline{\log x},$$ hence $$\hat \theta = \left( \overline{\log x} \right)^{-1} = \frac{n}{\sum_{i=1}^n \log x_i }.$$ This is plausible since the Pareto distribution has greater density in the tail when $\theta$ is small (this is obvious from inspecting the density); thus if the observations are large, the estimate of $\theta$ is small, but conversely, if the observations are close to $1$, the sum of their logarithms will be close to zero, yielding a large estimate of $\theta$ as this means the tail is relatively thin. Your estimator clearly cannot work because it cannot give $\hat \theta > 1$.


Regarding the actual question of the asymptotic variance of this estimator, we first set out to find the distribution of the random variable $Y = \log X$. Note this is a monotone transformation with the choice $g(x) = \log x$, hence we simply have $$f_Y(y) = f_X(g^{-1}(y)) \left| \frac{dg^{-1}}{dy} \right| = \theta \left( e^y \right)^{-\theta-1} e^y = \theta e^{-\theta y}, \quad y > 0.$$ We immediately recognize this as an exponential distribution with rate parameter $\theta$. Therefore, the random variable $$S = \sum_{i=1}^n \log X_i \sim \operatorname{Gamma}(n, \theta),$$ being the sum of $n$ iid exponential distributions, where the parametrization used throughout is a rate: $$f_S(s) = \frac{\theta^n s^{n-1} e^{-\theta s}}{\Gamma(n)}, \quad s > 0.$$ It easily follows that the distribution of the MLE estimator is $$\hat \theta = \frac{n}{S} \sim \operatorname{InvGamma}(n,n\theta),$$ with density $$f_{\hat\theta}(t) = \frac{(n \theta)^n e^{-n \theta/t}}{t^{n+1} \Gamma(n)}, \quad t > 0.$$ It is now simple to verify that the exact variance of the estimator is $$\operatorname{Var}[\hat\theta] = \frac{(n \theta)^2}{(n-1)^2 (n-2)}, \quad n > 2,$$ which asymptotically tends to $0$ as $n \to \infty$.

Moreover, we can use the above result to calculate the bias; clearly, for small sample sizes, $\hat \theta$ is biased, but for large sample sizes, this estimator has remarkably good properties as it is asymptotically unbiased, and its variance is $\mathcal O(n^{-1})$.

$\endgroup$
3
  • $\begingroup$ Thank you! Indeed I had a typo for the MLE estimator. I meant log of course. Your explanation is great. What a sharp insight to note that the transformation of the density yields a inverse gamma in the end. $\endgroup$ Apr 22, 2015 at 20:14
  • $\begingroup$ And like RadonNikodym points out, it should be $$InvGamma(n,\frac{n}{\theta})$$ $\endgroup$ Apr 22, 2015 at 20:36
  • $\begingroup$ The inverse gamma parameter is either $n\theta$ if it is scale, or $\frac{1}{n\theta}$ if it is rate, but it absolutely cannot be $n/\theta$ or $\theta/n$. I've verified the computation of the PDF of $\hat\theta$ as well as the variance by hand calculation of the transformation, as well as by computer simulation. $\endgroup$
    – heropup
    Apr 22, 2015 at 21:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.