Let $X_{1}, X_{2}, \ldots, X_{n}$ denote a random sample from an exponential distribution with density $$f(x)=\left\{\begin{aligned} \left(\frac{1}{\theta} \right)e^{-x/\theta}, x>0\\ 0, \quad \text{elsewhere}\end{aligned} \right.$$ Consider the following five estimators of $\theta$ $$\hat{\theta_{1}}=X_{1}, \quad \hat{\theta_{2}}=\frac{X_{1}+X_{2}}{2}, \quad \hat{\theta_{2}}=\frac{X_{1}+2X_{2}}{3}, \quad \hat{\theta_{4}}=\min(X_{1},X_{2},X_{3}), \quad \hat{\theta_{5}}=\overline{X}$$

  1. Which of these estimators are unbiased?

  2. Among the unbiased estimators, which has the smallest variance?

  1. I know that an estimator is unbiased if $\mathbb{E}[\hat{\theta_{i}}]=\theta$ so, by definition we can see that \begin{eqnarray*} \mathbb{E}[\hat{\theta_1}]&=&\int_{-\infty}^{\infty}xf(x)dx=\int_{0}^{\infty}x\left(\frac{1}{\theta}\right)e^{-x/\theta}dx=\theta \end{eqnarray*} \begin{eqnarray*} \mathbb{E}[\hat{\theta_2}]=\frac{1}{2}\mathbb{E}[X_1]+\frac{1}{2}\mathbb{E}[X_2]=\theta \end{eqnarray*}

\begin{eqnarray*} \mathbb{E}[\hat{\theta_2}]=\frac{1}{3}\mathbb{E}[X_1]+\frac{2}{3}\mathbb{E}[X_2]=\theta \end{eqnarray*}

\begin{eqnarray*} \mathbb{E}[\hat{\theta_5}]=\mathbb{E}\left[ \frac{1}{n}\sum_{1\leq i \leq n}X_{i}\right]=\frac{1}{n}\sum_{1\leq i \leq n}\mathbb{E}[X_{i}]=\frac{n}{n}\theta=\theta \end{eqnarray*}

Question 1: But how can I calculate $\mathbb{E}[\hat{\theta_4}]$?

b. Using the fact $\mathbb{V}[X_{i}]=\mathbb{E}[X^{2}_i]-\mathbb{E}[X_i]$ I obtained $\mathbb{V}[\hat{\theta_1}]=\theta^{2}, \mathbb{V}[\hat{\theta_2}]=\theta^{2}/2, \mathbb{V}[\hat{\theta}_3]=5\theta^{2}/9$.

Question 2: How can I calculate $\mathbb{V}[\hat{\theta}_4]$ and $\mathbb{V}[\hat{\theta}_5]$?


1 Answer 1


$$\Pr[\min(X_1, X_2, X_3) > x] = \Pr[(X_1 > x) \cap (X_2 > x) \cap (X_3 > x)],$$ because if the smallest of $X_1, X_2, X_3$ is greater than $x$, then each of them must be greater than $x$; conversely, if each is greater than $x$, then the smallest of them is also greater than $x$.

Then, by independence, the right-hand side is $$\Pr[X_1 > x]\Pr[X_2 > x]\Pr[X_3 > x],$$ and since the $X_i$ are identically distributed, this is $$(e^{-x/\theta})^3.$$ Therefore, $$\Pr[\min(X_1, X_2, X_3) \le x] = 1 - e^{-3x/\theta}.$$ Hence $\hat \theta_4$ is exponentially distributed with parameter $\theta/3$; so what is the expectation?

It should also be obvious how to compute the variance of $\hat \theta_4$. To compute the variance of $\hat \theta_5$, just use the linearity of variance when the $X_i$ are independent. In other words,

$$\operatorname{Var}\left[\frac{X_1 + X_2 + \cdots + X_n}{n} \right] = \frac{1}{n^2} \left(\operatorname{Var}[X_1] + \operatorname{Var}[X_2] + \cdots + \operatorname{Var}[X_n]\right) = \frac{n \operatorname{Var}[X]}{n^2}.$$ I leave you to compute the rest. Note that the question of which estimator has the smallest variance depends on the choice of $n$.

  • $\begingroup$ Thank you so much! Excellent explanation. $\endgroup$
    – user798113
    Nov 19, 2020 at 4:44

You must log in to answer this question.