# Derive an unbiased estimator for $\theta$.

Exercise: Let $X_1,\ldots,X_n$ be a random sample from the distribution with density $$f(x|\theta) = \dfrac{2x}{\theta^2}\mathbb{1}_{(0,\theta)}(x)$$ w.r.t. the Lebesgue measure. Derive an unbiased estimator for $\theta$.

What I've tried: I need to find an estimator $\delta(X_1,\ldots,X_n)$ such that $\operatorname{E}\big[\delta(X_1,\ldots,X_n)\big] = \theta$. I tried the maximum likelihood estimator but that got me nowhere. The log-likelihood would be equal to $\log L = \log\prod\limits_{i = 1}^n\dfrac{2x_i}{\theta^2}\mathbb{1}_{(0,\theta)}(x_i) = \sum\limits_{i= 1}^n2x_i - 2n\log\theta$ (I'm not sure if I can leave the indicator function like this btw). If I now take the derivate w.r.t. $\theta$ and set it to zero I get that $\hat{\theta}_{ML} = 0$, which is not an unbiased estimator.

Question: How do I solve this exercise? In general; what would a efficient approach be to find an unbiased estimator in exercises like this one?

Thanks in advance!

• I would start with exploring $\mathbb EX_1^r$. Note that $\mathbb EX_1=\frac23\theta$ so that $\frac3{2n}(X_1+\cdots+X_n)$ is an unbiased estimator of $\theta$. – drhab Jan 18 '18 at 12:45

## 2 Answers

In general, note that maximum likelihood estimators are not necessarily unbiased estimators.

I'm not familiar with Lebesgue integration, but hopefully using non-measure theoretic tools can help you find this.

First of all, observe that $$\mathbb{E}[X_1]=\dfrac{2}{\theta^2}\int_{0}^{\theta}x^2\text{ d}x=\dfrac{2}{\theta^2}\cdot\dfrac{\theta^3}{3}=\dfrac{2\theta}{3}\text{.}$$ Thus, the estimator $$\hat{\theta}=\dfrac{3}{2n}\sum_{i=1}^{n}X_i$$ is unbiased for $\theta$, since $$\mathbb{E}[\hat{\theta}]=\dfrac{3}{2n}\sum_{i=1}^{n}\mathbb{E}[X_i]=\dfrac{3}{2n}\cdot \dfrac{2\theta}{3}\cdot n = \theta\text{.}$$

• Hope I can revive this question a bit. Could you eloborate what the thought process behind your steps was and in what situation this would not work? For example, what would the result of $\mathbb{E}[X_{1}]$ have to be to not lead to an estimator? What was the initial idea behind starting with computing the $\mathbb{E}[X_{1}]$? Hope you can help! – S. Crim Nov 28 '18 at 19:38
• @S.Crim The underlying idea is simply that since $X_1, \dots, X_n$ are iid that $$\mathbb{E}\left[\dfrac{1}{n}\sum_{i=1}^{n}f(X_i)\right]= \dfrac{1}{n}\sum_{i=1}^{n}\mathbb{E}[f(X_i)] = \dfrac{1}{n}\cdot n\mathbb{E}[f(X_1)] = \mathbb{E}[f(X_1)]$$ hence, $\dfrac{1}{n}\sum_{i=1}^{n}f(X_i)$ is an unbiased estimator of $\mathbb{E}[f(X_1)]$, and by linearity of $\mathbb{E}$, we could multiply both sides by the same constant to obtain that $\dfrac{3}{2n}\sum_{i=1}^{n}X_i$ is an unbiased estimator of $\dfrac{3}{2}\mathbb{E}[X_1] = \theta$. This all relies on existence of $\mathbb{E}[X_1]$. – Clarinetist Nov 28 '18 at 19:50
• Ah, perfect! So this approach fails when $\mathbb{E}[X_{1}]$ doesn't exist? Thanks alot for the help. – S. Crim Nov 28 '18 at 19:54
• @S.Crim Yes, it does fail. It would be an interesting exercise to, say, experiment with the Cauchy distribution (for which the expected value is infinite). I found this on a quick Wikipedia search, but haven't read it yet: en.wikipedia.org/wiki/… – Clarinetist Nov 28 '18 at 19:56

The likelihood function is\begin{align*} L(θ; x_1, \cdots, x_n) &= \frac{2^n}{θ^{2n}} \prod_{k = 1}^n x_k \prod_{k = 1}^n I_{(0, θ)}(x_k)\\ &= \frac{2^n}{θ^{2n}} \prod_{k = 1}^n x_k I_{(0, +\infty)}\left( \min_{1 \leqslant k \leqslant n} x_k \right) I_{(-\infty, θ)}\left( \max_{1 \leqslant k \leqslant n} x_k \right). \end{align*} For fixed $x_1, \cdots, x_n$, $L(θ; x_1, \cdots, x_n) = 0$ for $θ < \max\limits_{1 \leqslant k \leqslant n} x_k$, and $L(θ; x_1, \cdots, x_n)$ is decreasing with respect to $θ$ for $θ > \max\limits_{1 \leqslant k \leqslant n} x_k$. Thus the MLE of $θ$ is$$\hat{θ}(X_1, \cdots, X_n) = \max_{1 \leqslant k \leqslant n} X_k.$$

Note that the density function of $\max\limits_{1 \leqslant k \leqslant n} X_k$ is$$f_n(x; θ) = n (F(x; θ))^{n - 1} f(x; θ) = \frac{2n}{θ^{2n}} x^{2n - 1} I_{(0, θ)}(x),$$ thus$$E_θ(\hat{θ}) = \int_0^θ x \cdot \frac{2n}{θ^{2n}} x^{2n - 1} \,\mathrm{d}x = \frac{2n}{2n + 1} θ.$$ Therefore, an unbiased estimator of $θ$ is $\displaystyle \frac{2n + 1}{2n} \max\limits_{1 \leqslant k \leqslant n} X_k$.

• In fact, because $\max\limits_{1 \leqslant k \leqslant n} X_k$ is a sufficient statistic, this unbiased estimator is also a UMVUE. – Saad Jan 18 '18 at 12:57
• Thanks for your reply! Could you explain to me why the density function of$T(X)$ equals $f_n(x;\theta) = n(F(x;\theta))^{n-1}f(x;\theta)$? – titusAdam Jan 20 '18 at 13:51
• @titusAdam This is a property of order statistics. You could refer to The joint distribution of the order statistics of an absolutely continuous distribution. – Saad Jan 20 '18 at 14:15
• Thanks again! However, how do you know that $\max(X_1,\ldots,X_n)$ is the $n-th$ order statistic? More specifically, how do you know that it's not $f_k(x;\theta)$ that is the density function of $\max X_k$? – titusAdam Jan 21 '18 at 10:44
• The definition of order statistics is right in the first part of the wiki article saying $X_{(n)} = \max\limits_{1 \leqslant k \leqslant n} X_k$. – Saad Jan 21 '18 at 10:49