# A problem about the estimations of a parameter

Let $$X_{1}, X_{2}, \ldots, X_{n}$$ denote a random sample from an exponential distribution with density f(x)=\left\{\begin{aligned} \left(\frac{1}{\theta} \right)e^{-x/\theta}, x>0\\ 0, \quad \text{elsewhere}\end{aligned} \right. Consider the following five estimators of $$\theta$$ $$\hat{\theta_{1}}=X_{1}, \quad \hat{\theta_{2}}=\frac{X_{1}+X_{2}}{2}, \quad \hat{\theta_{2}}=\frac{X_{1}+2X_{2}}{3}, \quad \hat{\theta_{4}}=\min(X_{1},X_{2},X_{3}), \quad \hat{\theta_{5}}=\overline{X}$$

1. Which of these estimators are unbiased?

2. Among the unbiased estimators, which has the smallest variance?

1. I know that an estimator is unbiased if $$\mathbb{E}[\hat{\theta_{i}}]=\theta$$ so, by definition we can see that $$\begin{eqnarray*} \mathbb{E}[\hat{\theta_1}]&=&\int_{-\infty}^{\infty}xf(x)dx=\int_{0}^{\infty}x\left(\frac{1}{\theta}\right)e^{-x/\theta}dx=\theta \end{eqnarray*}$$ $$\begin{eqnarray*} \mathbb{E}[\hat{\theta_2}]=\frac{1}{2}\mathbb{E}[X_1]+\frac{1}{2}\mathbb{E}[X_2]=\theta \end{eqnarray*}$$

$$\begin{eqnarray*} \mathbb{E}[\hat{\theta_2}]=\frac{1}{3}\mathbb{E}[X_1]+\frac{2}{3}\mathbb{E}[X_2]=\theta \end{eqnarray*}$$

$$\begin{eqnarray*} \mathbb{E}[\hat{\theta_5}]=\mathbb{E}\left[ \frac{1}{n}\sum_{1\leq i \leq n}X_{i}\right]=\frac{1}{n}\sum_{1\leq i \leq n}\mathbb{E}[X_{i}]=\frac{n}{n}\theta=\theta \end{eqnarray*}$$

Question 1: But how can I calculate $$\mathbb{E}[\hat{\theta_4}]$$?

b. Using the fact $$\mathbb{V}[X_{i}]=\mathbb{E}[X^{2}_i]-\mathbb{E}[X_i]$$ I obtained $$\mathbb{V}[\hat{\theta_1}]=\theta^{2}, \mathbb{V}[\hat{\theta_2}]=\theta^{2}/2, \mathbb{V}[\hat{\theta}_3]=5\theta^{2}/9$$.

Question 2: How can I calculate $$\mathbb{V}[\hat{\theta}_4]$$ and $$\mathbb{V}[\hat{\theta}_5]$$?

$$\Pr[\min(X_1, X_2, X_3) > x] = \Pr[(X_1 > x) \cap (X_2 > x) \cap (X_3 > x)],$$ because if the smallest of $$X_1, X_2, X_3$$ is greater than $$x$$, then each of them must be greater than $$x$$; conversely, if each is greater than $$x$$, then the smallest of them is also greater than $$x$$.
Then, by independence, the right-hand side is $$\Pr[X_1 > x]\Pr[X_2 > x]\Pr[X_3 > x],$$ and since the $$X_i$$ are identically distributed, this is $$(e^{-x/\theta})^3.$$ Therefore, $$\Pr[\min(X_1, X_2, X_3) \le x] = 1 - e^{-3x/\theta}.$$ Hence $$\hat \theta_4$$ is exponentially distributed with parameter $$\theta/3$$; so what is the expectation?
It should also be obvious how to compute the variance of $$\hat \theta_4$$. To compute the variance of $$\hat \theta_5$$, just use the linearity of variance when the $$X_i$$ are independent. In other words,
$$\operatorname{Var}\left[\frac{X_1 + X_2 + \cdots + X_n}{n} \right] = \frac{1}{n^2} \left(\operatorname{Var}[X_1] + \operatorname{Var}[X_2] + \cdots + \operatorname{Var}[X_n]\right) = \frac{n \operatorname{Var}[X]}{n^2}.$$ I leave you to compute the rest. Note that the question of which estimator has the smallest variance depends on the choice of $$n$$.