Derive the bias and MSE of the estimator $\hat{\beta}$

Question

Let $Y_1, Y_2, \ldots, Y_n$ denote a random sample of size $n$ from a population whose density is given by $$ f(y)= \begin{cases} 3\beta^3 y^{-4} & \beta \leq y \\ 0 & \text{elsewhere} \end{cases} $$ where $\beta > 0$ is unknown. (This is a Pareto distribution) Consider the estimator $\hat{\beta} = \min(Y_1, Y_2, \ldots, Y_n)$.

a. Derive the bias of the estimator $\hat{\beta}$.

b. Derive MSE($\hat{\beta}$).

For part a

Since this is an estimator that is dealing with random variables, and I only need the minimum, then I can represent this as ordered statistics. Namely, $\hat{\beta} = Y_{(1)}$.

Using the formula for ordered statistics, the pdf is: $$ g_{(1)}(Y_{(1)}) = n(1-F(y))^{n-1}f(y) $$ My first question arises here, I am not sure how I am supposed to find $F(y)$ (and I am assuming that $f(y)$ is the given distribution above).

Checking against the solution, I know that the pdf is supposed to be: $$ g_{(1)}(Y_{(1)}) = 3n\beta^{3n}y^{(-3n-1)} $$

After solving getting the pdf of $Y_{(1)}$, I can then use it in the bias formula.

$$ Bias(\hat{\beta}) = E[\hat{\beta}] - \beta \\ Bias(\hat{\beta}) = E[3n\beta^{3n}y^{(-3n-1)}] - \beta $$ My second question arises here. I'm not really sure how I am supposed to find the expectation. Namely, I don't know how I am supposed to treat the $y$ variable. I think that the $\beta$ variable is supposed to be treated the same way as a random variable, and that $n$ is supposed to be treated as a constant.

After solving for the expectation, I should be able to get the Bias.

For part b $$ MSE(\hat{\beta}) = Bias(\hat{\beta})^2 + Var(\hat{\beta}) $$ The bias part of this formula should follow from part a. This would leave just the variance to calculate. I would expect to use the variance formula: $$ Var(\hat{\beta}) = E[\hat{\beta}^2] - E[\hat{\beta}]^2 $$ And I would also expect that both expectations in this formula could be calculated using the same methods in part a. Is this correct?

Answer 1

I got this question recently. This question is from Mathematical Statistics - Wackerly - Exercise 8.1.

Let $Y_{1}, Y_{2}, \ldots, Y_{n}$ denote a random sample of size $n$ from a population whose density is given by $$ f(y)=\left\{\begin{array}{ll} 3 \beta^{3} y^{-4} & , \quad \beta \leq y \\ 0 & , \text { en cualquier otro caso } \end{array}\right. $$ where $\beta>0$ is unknown. (This is one of the Pareto distributions introduced in Exercise 6.18.) Consider the estimator $\hat{\beta}=\min \left(Y_{1}, Y_{2}, \ldots, Y_{n}\right)$

This my approach:

a.

Considering distribution function from Pareto distribution we have:

$$F(y)=\begin{cases}0,& y<\beta\\1-\left(\frac{\beta}{y}\right)^\alpha, & y\geq \beta \end{cases}$$

Which density function (derivative) is: $$f(y)=\begin{cases}0,& y<\beta\\ \alpha \beta ^\alpha y^{-(\alpha +1)}, & y\geq \beta \end{cases}$$

We proceed to calculate $Y_{(1)}$ density function (with $\alpha=3$, our problem):

\begin{align} \begin{split} g_{(1)}(y)&=n\left[1-F(y)\right]^{n-1}f(y) \\ &=n\left[1-\left( 1-\left(\frac{\beta}{y}\right)^3 \right)\right]^{n-1} 3 \beta ^3 y^{-(3+1)}\\ &= n\left[\beta^{3(n-1)}y^{-3(n-1)}\right]3 \beta ^3 y^{-(4)}\\ &= 3n\beta^{3n}y^{-(3n+1)}, \qquad y\geq \beta \end{split} \end{align}

Now, we'll check this definition:

Let $\hat{\theta}$ be a point estimator for a parameter $\theta$. Then $\hat{\theta}$ is an unbiased estimator if $E(\hat{\theta})=\theta$. If $E(\hat{\theta})\neq \theta$, $\hat{\theta}$ is biased.

\begin{align*} E(Y_{(1)}) &= \int_{\infty}^{\infty} y f_{(1)}(y) \ d y\\ &= \int_{\beta}^{\infty} y 3n\beta^{3n}y^{-(3n+1)} \ dy\\ &= 3n\beta^{3n}\lim_{h\to \infty}\int_{\beta}^{h} y^{-3n} \ dy\\ &= 3n\beta^{3n}\lim_{h\to \infty}\left[\frac{y^{-3n+1}}{-3n+1}\right]_\beta^h\\ &= 3n\beta^{3n}\left[-\frac{\beta^{-3n+1}}{-3n+1}\right]\\ &= \left(\frac{3n}{3n-1}\right)\beta \end{align*}

So we know, by definition, $\hat{\theta}$ is a biased estimator. Now we were asked to find the bias, which is easily to calculate considering this definition:

$$B(\hat{\theta}) = E(\hat{\theta})-\theta$$

Therefore:

\begin{align*} B(\hat{\beta})&=E(\hat{\beta})-\beta\\ &= \left(\frac{3n}{3n-1}\right)\beta - \beta \\ &= \left(\frac{1}{3n-1}\right) \end{align*}

b.

We proceed to calculate $E(\hat{\beta}^2)$ (which later will be necessary): \begin{align*} E(\hat{\beta}^2)= E(Y_{(1)}^2) &= \int_{\infty}^{\infty} y^2 f_{(1)}(y) \ d y\\ &= \int_{\beta}^{\infty} y^2 3n\beta^{3n}y^{-(3n+1)} \ dy\\ &= 3n\beta^{3n}\lim_{h\to \infty}\int_{\beta}^{h} y^{-3n+1} \ dy\\ &= 3n\beta^{3n}\lim_{h\to \infty}\left[\frac{y^{-3n+2}}{-3n+2}\right]_\beta^h\\ &= 3n\beta^{3n}\left[-\frac{\beta^{-3n+2}}{-3n+2}\right]\\ &= \left(\frac{3n}{3n-2}\right)\beta^2 \end{align*}

Finally, we have:

\begin{align*} MSE(\hat{\beta})&=E[(\hat{\beta}-\beta)^2]\\ &= E[\hat{\beta}^2-2\hat{\beta}\beta +\beta^2]\\ &= E(\hat{\beta}^2)-2\beta E(\hat{\beta})+ E(\beta^2)\\ &= E(\hat{\beta}^2)-2\beta E(\hat{\beta})+ \beta^2\\ &= \left(\frac{3n}{3n-2}\right)\beta^2 -2\beta^2\left(\frac{3n}{3n-1}\right) +\beta^2\\ &= \left(\frac{3n}{3n-2}\right)\beta^2 -\left(\frac{3n}{3n-1}\right)\beta^2\\ &= \left(\frac{3n}{3n-2}-\frac{3n}{3n-1}\right)\beta^2\\ &= \left(\frac{2}{(3n-1)(3n-2)}\right)\beta^2 \end{align*}

A nicest version of this problem (in Spanish) is at this repository on Github: Parcial 3 - Estadística Matemática.