2
$\begingroup$

$$\mathbf{X} = \{X_1,X_2,\dots,X_n\}$$ sequence of i.i.d. RV's. Let the distribution of the RV's be defined by $$f(x|\theta)=\frac{\theta}{x^{\theta+1}}, \quad x>1, \quad \theta>1$$ I am interested in the asymptotic variance of the MLE for $\theta$. That is; $$\lim_{n\rightarrow \infty}\mathbb{V}ar_{\theta^*}(\hat{\theta}_n) = \lim_{n\rightarrow \infty}\mathbb{V}ar_{\theta^*}\left( \frac{n}{\sum_{i=1}^n X_i}\right)$$

What I thought is to use the delta-method (1-st order) approximation: $$\mathbb{V}ar_{\theta^*}(\hat{\theta}_n(\mathbf{X})) \approx \left(\hat{\theta}_n\left(\mathbb{E}_{\theta^*}[\mathbf{X}]\right) \right)^2\mathbb{V}ar_{\theta^*}(\mathbf{X})$$

In the above, subscript $\theta^*$ means with respect to the density defined by the true parameter value $\theta^*$ (assuming convergence in probability). Also, the expected value of one RV equals:

$$\mathbb{E}_{\theta^*}[X]=\frac{\theta}{1+\theta}$$ If I'm correct. Would the calculation of the asymptotic variance be possible using the delta-method approximation? I could not get far at all.

$\endgroup$
1
  • $\begingroup$ Why do you think the variance exists? $\endgroup$
    – Did
    Apr 21, 2015 at 6:47

1 Answer 1

1
$\begingroup$

Your MLE for $\theta$ does not seem correct. If $\boldsymbol x = (x_1, \ldots, x_n)$ is an iid sample drawn from a $X \sim \operatorname{Pareto}(1,\theta)$ distribution with density $$f_X(x) = \theta x^{-\theta-1} \mathbb{1}(x > 1)$$ then the likelihood function is $$\mathcal L (\theta \mid \boldsymbol x) = \theta^n \left(\prod_{i=1}^n x_i \right)^{-\theta-1} \mathbb{1}(x_{(1)} > 1).$$ The log-likelihood is then $$\ell(\theta \mid \boldsymbol x) = n \log \theta - (\theta+1) \sum_{i=1}^n \log x_i + \log \mathbb{1}(x_{(1)} > 1),$$ which suggests letting $\overline{ \log x} = \frac{1}{n} \sum_{i=1}^n \log x_i$. Solving for the critical points gives $$0 = \frac{\partial \ell}{\partial \theta} = \frac{n}{\theta} - n \, \overline{\log x},$$ hence $$\hat \theta = \left( \overline{\log x} \right)^{-1} = \frac{n}{\sum_{i=1}^n \log x_i }.$$ This is plausible since the Pareto distribution has greater density in the tail when $\theta$ is small (this is obvious from inspecting the density); thus if the observations are large, the estimate of $\theta$ is small, but conversely, if the observations are close to $1$, the sum of their logarithms will be close to zero, yielding a large estimate of $\theta$ as this means the tail is relatively thin. Your estimator clearly cannot work because it cannot give $\hat \theta > 1$.


Regarding the actual question of the asymptotic variance of this estimator, we first set out to find the distribution of the random variable $Y = \log X$. Note this is a monotone transformation with the choice $g(x) = \log x$, hence we simply have $$f_Y(y) = f_X(g^{-1}(y)) \left| \frac{dg^{-1}}{dy} \right| = \theta \left( e^y \right)^{-\theta-1} e^y = \theta e^{-\theta y}, \quad y > 0.$$ We immediately recognize this as an exponential distribution with rate parameter $\theta$. Therefore, the random variable $$S = \sum_{i=1}^n \log X_i \sim \operatorname{Gamma}(n, \theta),$$ being the sum of $n$ iid exponential distributions, where the parametrization used throughout is a rate: $$f_S(s) = \frac{\theta^n s^{n-1} e^{-\theta s}}{\Gamma(n)}, \quad s > 0.$$ It easily follows that the distribution of the MLE estimator is $$\hat \theta = \frac{n}{S} \sim \operatorname{InvGamma}(n,n\theta),$$ with density $$f_{\hat\theta}(t) = \frac{(n \theta)^n e^{-n \theta/t}}{t^{n+1} \Gamma(n)}, \quad t > 0.$$ It is now simple to verify that the exact variance of the estimator is $$\operatorname{Var}[\hat\theta] = \frac{(n \theta)^2}{(n-1)^2 (n-2)}, \quad n > 2,$$ which asymptotically tends to $0$ as $n \to \infty$.

Moreover, we can use the above result to calculate the bias; clearly, for small sample sizes, $\hat \theta$ is biased, but for large sample sizes, this estimator has remarkably good properties as it is asymptotically unbiased, and its variance is $\mathcal O(n^{-1})$.

$\endgroup$
3
  • $\begingroup$ Thank you! Indeed I had a typo for the MLE estimator. I meant log of course. Your explanation is great. What a sharp insight to note that the transformation of the density yields a inverse gamma in the end. $\endgroup$ Apr 22, 2015 at 20:14
  • $\begingroup$ And like RadonNikodym points out, it should be $$InvGamma(n,\frac{n}{\theta})$$ $\endgroup$ Apr 22, 2015 at 20:36
  • $\begingroup$ The inverse gamma parameter is either $n\theta$ if it is scale, or $\frac{1}{n\theta}$ if it is rate, but it absolutely cannot be $n/\theta$ or $\theta/n$. I've verified the computation of the PDF of $\hat\theta$ as well as the variance by hand calculation of the transformation, as well as by computer simulation. $\endgroup$
    – heropup
    Apr 22, 2015 at 21:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .