Suppose that we have a sample $X_1,\ldots,X_n$ from the distribution with density function $$f(x\mid\theta) = \dfrac{2x}{\theta^2}\mathbb{1}_{(0,\theta)}(x)$$ w.r.t. the Lebesgue measure. I recently learned that if we use $T(X) =\max(X_1,\ldots,X_n)$ as an estimator for $\theta$, the density function of $T(X)$ equals $$f_n(x\mid\theta) = n(F(x\mid\theta))^{n-1}f(x\mid\theta) = \dfrac{2n}{\theta^{2n}}x^{n-1}\mathbb{1}_{(0,\theta)}(x)$$ and thus that $$\operatorname{E}_\theta(T(X)) = \displaystyle\int_0^\theta x\dfrac{2n}{\theta^{2n}}x^{n-1}\mathbb{1}_{(0,\theta)}(x) \, dx.$$ What I don't understand is when the density function of an estimator differs from the density function of the data. In my book the following example is given:

Suppose $X\sim\operatorname{Geom}(\theta)$, with $\theta\in(0,1)$. We seek for an unbiased estimator for $\theta$. We have that $$\operatorname{E}_\theta d(X) = \sum\limits_{i = 1}^\infty d(i)\theta(1-\theta)^{i-1}$$ where $d(X)$ is an estimator for $\theta$. Here the density function of the data is used to calculate the expected value of the estimator. Why is this case different to the case with the order statistic? This computation of the expected value would surely be wrong if $d(X)$ equals the order statistic right?

Question: When calculating the expected value of an estimator, how do you determine which density function you should use? Is the order statistic a special case? I would really like to know what the underlying theory behind all this is, because now it comes across as quite random.

Thanks in advance!


You can always use the distribution of the "data" in order to compute the expected value of an estimator, however mostly this is inconvenient as it involves repeated sums of sums or integrals. Thus, if it worth to derive the distribution of the estimator itself and use it for computations. E.g., you have two i.i.d $X_1, X_2$ from $N(\mu, \sigma^2)$ and you are interested in estimating $\mu$. So, instead of using the fact that $$ \bar{X}_2 \sim N(\mu, \sigma^2/2), $$
you decided to compute $\mathbb{E}\bar{X}_2$ directly. Therefore, $\bar{X}_2 = g(X_1, X_2)$, hence \begin{align} \mathbb{E} \bar{X}_2 &= \mathbb{E} g(X_1, X_2)\\ & = \int\int g(x_1, x_2) f_{X_1,X_2}(x_1, x_2)dx_1dx_2 \\ &= \frac{1}{2}\int\int (x_1 + x_2) f_{X_1,X_2}(x_1, x_2)dx_1dx_2\\ &= \frac{1}{2}\int\int x_1 f_{X_1}(x_1)f_{X_2}(x_2)dx_1dx_2 + \frac{1}{2}\int\int x_2 f_{X_1}(x_1)f_{X_2}(x_2)dx_1dx_2\\ & = \frac{1}{2}\mathbb{E}X_1 + \frac{1}{2}\mathbb{E}X_2 \\ & = \mu. \end{align}

  • $\begingroup$ Thanks for your reply. However, why is it correct then to calculate the expected value of $d(X)$ as is done in the example of my book? $\endgroup$ – titusAdam Jan 22 '18 at 8:09
  • $\begingroup$ Well, if $X\sim Geo(p)$, and your estimator is some function $\phi(X)$, thus $\mathbb{E}\phi(X) = \sum \phi(x) q^{x-1}p $. $\endgroup$ – V. Vancak Jan 22 '18 at 10:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.