2
$\begingroup$

Exercise: Let $X_1,\ldots,X_n$ be a random sample from the distribution with density $$f(x|\theta) = \dfrac{2x}{\theta^2}\mathbb{1}_{(0,\theta)}(x)$$ w.r.t. the Lebesgue measure. Derive an unbiased estimator for $\theta$.

What I've tried: I need to find an estimator $\delta(X_1,\ldots,X_n)$ such that $\operatorname{E}\big[\delta(X_1,\ldots,X_n)\big] = \theta$. I tried the maximum likelihood estimator but that got me nowhere. The log-likelihood would be equal to $\log L = \log\prod\limits_{i = 1}^n\dfrac{2x_i}{\theta^2}\mathbb{1}_{(0,\theta)}(x_i) = \sum\limits_{i= 1}^n2x_i - 2n\log\theta$ (I'm not sure if I can leave the indicator function like this btw). If I now take the derivate w.r.t. $\theta$ and set it to zero I get that $\hat{\theta}_{ML} = 0$, which is not an unbiased estimator.

Question: How do I solve this exercise? In general; what would a efficient approach be to find an unbiased estimator in exercises like this one?

Thanks in advance!

$\endgroup$
  • $\begingroup$ I would start with exploring $\mathbb EX_1^r$. Note that $\mathbb EX_1=\frac23\theta$ so that $\frac3{2n}(X_1+\cdots+X_n)$ is an unbiased estimator of $\theta$. $\endgroup$ – drhab Jan 18 '18 at 12:45
5
$\begingroup$

In general, note that maximum likelihood estimators are not necessarily unbiased estimators.

I'm not familiar with Lebesgue integration, but hopefully using non-measure theoretic tools can help you find this.

First of all, observe that $$\mathbb{E}[X_1]=\dfrac{2}{\theta^2}\int_{0}^{\theta}x^2\text{ d}x=\dfrac{2}{\theta^2}\cdot\dfrac{\theta^3}{3}=\dfrac{2\theta}{3}\text{.}$$ Thus, the estimator $$\hat{\theta}=\dfrac{3}{2n}\sum_{i=1}^{n}X_i$$ is unbiased for $\theta$, since $$\mathbb{E}[\hat{\theta}]=\dfrac{3}{2n}\sum_{i=1}^{n}\mathbb{E}[X_i]=\dfrac{3}{2n}\cdot \dfrac{2\theta}{3}\cdot n = \theta\text{.}$$

$\endgroup$
  • $\begingroup$ Hope I can revive this question a bit. Could you eloborate what the thought process behind your steps was and in what situation this would not work? For example, what would the result of $\mathbb{E}[X_{1}]$ have to be to not lead to an estimator? What was the initial idea behind starting with computing the $\mathbb{E}[X_{1}]$? Hope you can help! $\endgroup$ – S. Crim Nov 28 '18 at 19:38
  • 1
    $\begingroup$ @S.Crim The underlying idea is simply that since $X_1, \dots, X_n$ are iid that $$\mathbb{E}\left[\dfrac{1}{n}\sum_{i=1}^{n}f(X_i)\right]= \dfrac{1}{n}\sum_{i=1}^{n}\mathbb{E}[f(X_i)] = \dfrac{1}{n}\cdot n\mathbb{E}[f(X_1)] = \mathbb{E}[f(X_1)]$$ hence, $\dfrac{1}{n}\sum_{i=1}^{n}f(X_i)$ is an unbiased estimator of $\mathbb{E}[f(X_1)]$, and by linearity of $\mathbb{E}$, we could multiply both sides by the same constant to obtain that $\dfrac{3}{2n}\sum_{i=1}^{n}X_i$ is an unbiased estimator of $\dfrac{3}{2}\mathbb{E}[X_1] = \theta$. This all relies on existence of $\mathbb{E}[X_1]$. $\endgroup$ – Clarinetist Nov 28 '18 at 19:50
  • $\begingroup$ Ah, perfect! So this approach fails when $\mathbb{E}[X_{1}]$ doesn't exist? Thanks alot for the help. $\endgroup$ – S. Crim Nov 28 '18 at 19:54
  • 1
    $\begingroup$ @S.Crim Yes, it does fail. It would be an interesting exercise to, say, experiment with the Cauchy distribution (for which the expected value is infinite). I found this on a quick Wikipedia search, but haven't read it yet: en.wikipedia.org/wiki/… $\endgroup$ – Clarinetist Nov 28 '18 at 19:56
3
$\begingroup$

The likelihood function is\begin{align*} L(θ; x_1, \cdots, x_n) &= \frac{2^n}{θ^{2n}} \prod_{k = 1}^n x_k \prod_{k = 1}^n I_{(0, θ)}(x_k)\\ &= \frac{2^n}{θ^{2n}} \prod_{k = 1}^n x_k I_{(0, +\infty)}\left( \min_{1 \leqslant k \leqslant n} x_k \right) I_{(-\infty, θ)}\left( \max_{1 \leqslant k \leqslant n} x_k \right). \end{align*} For fixed $x_1, \cdots, x_n$, $L(θ; x_1, \cdots, x_n) = 0$ for $θ < \max\limits_{1 \leqslant k \leqslant n} x_k$, and $L(θ; x_1, \cdots, x_n)$ is decreasing with respect to $θ$ for $θ > \max\limits_{1 \leqslant k \leqslant n} x_k$. Thus the MLE of $θ$ is$$ \hat{θ}(X_1, \cdots, X_n) = \max_{1 \leqslant k \leqslant n} X_k. $$

Note that the density function of $\max\limits_{1 \leqslant k \leqslant n} X_k$ is$$ f_n(x; θ) = n (F(x; θ))^{n - 1} f(x; θ) = \frac{2n}{θ^{2n}} x^{2n - 1} I_{(0, θ)}(x), $$ thus$$ E_θ(\hat{θ}) = \int_0^θ x \cdot \frac{2n}{θ^{2n}} x^{2n - 1} \,\mathrm{d}x = \frac{2n}{2n + 1} θ. $$ Therefore, an unbiased estimator of $θ$ is $\displaystyle \frac{2n + 1}{2n} \max\limits_{1 \leqslant k \leqslant n} X_k$.

$\endgroup$
  • $\begingroup$ In fact, because $\max\limits_{1 \leqslant k \leqslant n} X_k$ is a sufficient statistic, this unbiased estimator is also a UMVUE. $\endgroup$ – Saad Jan 18 '18 at 12:57
  • $\begingroup$ Thanks for your reply! Could you explain to me why the density function of$T(X)$ equals $f_n(x;\theta) = n(F(x;\theta))^{n-1}f(x;\theta)$? $\endgroup$ – titusAdam Jan 20 '18 at 13:51
  • $\begingroup$ @titusAdam This is a property of order statistics. You could refer to The joint distribution of the order statistics of an absolutely continuous distribution. $\endgroup$ – Saad Jan 20 '18 at 14:15
  • $\begingroup$ Thanks again! However, how do you know that $\max(X_1,\ldots,X_n)$ is the $n-th$ order statistic? More specifically, how do you know that it's not $f_k(x;\theta)$ that is the density function of $\max X_k$? $\endgroup$ – titusAdam Jan 21 '18 at 10:44
  • 1
    $\begingroup$ The definition of order statistics is right in the first part of the wiki article saying $X_{(n)} = \max\limits_{1 \leqslant k \leqslant n} X_k$. $\endgroup$ – Saad Jan 21 '18 at 10:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.