1
$\begingroup$

A given statistic : $T_c = \sum_{j=1}^n \frac{{(X_j - \bar X)}^2}{c}$, where $c$ is a constant, as an estimator of variance $\sigma^2.$

$X_1,\ldots, X_n$ denote a random sample from a population which has normal distribution with unknown mean $\mu$ and unknown variance $\sigma^2$.

The statistic is distributed as $x^2_{n-1}$ (a chi-squared variate with $n-1$ degrees of freedom).

I am tasked to find the bias of $T_c$.

I know the formula for bias is $\mathbb E \hat \theta - \theta$.

I found $\theta$ as $\mu = n - 1$ for a chi-squared distribution of $n-1$ degrees of freedom.

However, I am confused as to how to calculate $\mathbb E \hat \theta$.

What i thought of doing is to calculate $\mathbb E T_c$, but i got stuck halfway through, so I am not sure if I am doing the right thing. Any help please. thanks in advance.

$\endgroup$
3
  • $\begingroup$ @StubbornAtom ah sorry just added the squared bracket for my $T_c$ $\endgroup$ – Wei Xiong Yeo Sep 1 '18 at 13:07
  • $\begingroup$ Are you sure $T_c$ has distribution $\mathcal{X}^2_{n-1}$. Or did you mean to say that the $X_i$'s are $\mathcal{X}^2_{n-1}$ ? Also are the $X_i$'s independent ? $\endgroup$ – Digitalis Sep 1 '18 at 18:11
  • $\begingroup$ @Digitalis Yes I would assume they are independent. Also, I have edited my original post to include the distribution of $X_i$'s $\endgroup$ – Wei Xiong Yeo Sep 2 '18 at 3:43
0
$\begingroup$

Since $T_c$ is an estimator of $\sigma^2$. $T_c$ will be unbiased for $\theta = \sigma^2$ if $\mathbb{E}_{\mu,\sigma^2}(T_c) = \sigma^2 \quad \forall \mu, \sigma^2 \in \mathbb{R} \times \mathbb{R}_0^+$

\begin{align*} \mathbb{E}(T_c)&= \frac{1}{c}\mathbb{E} \Big( \sum_{j=1}^n X_j^2 + \bar{X}^2 -2X_j \bar{X} \Big) \\ &= \frac{1}{c} \sum_{j=1}^n \mathbb{E}(X_j^2) + \mathbb{E}(\bar{X}^2) - 2 \mathbb{E} (X_j \bar{X}) \end{align*}

Now calculate $\mathbb{E}(X_j^2),\mathbb{E}(\bar{X}^2)$ & $ \mathbb{E} (X_j \bar{X})$.

1. Using Var$(X) = \mathbb{E}(X^2) - \mathbb{E}(X)^2$ we find $\mathbb{E}(X_j^2) = \sigma^2 + \mu^2$.

2. \begin{align*} \mathbb{E}(\bar{X}^2) &= \mathbb{E}\Big( \big(\frac{1}{n} \sum_i^n X_i\big) \big(\frac{1}{n} \sum_i^n X_i\big)\Big) \\ &= \frac{1}{n^2} \mathbb{E}\Big( \big(\sum_i^n X_i\big) \big(\sum_i^n X_i\big)\Big)\\ &= \frac{1}{n^2} \mathbb{E}\Big( \sum_{i \neq j} X_iX_j + \sum_i X_i^2\Big) \\ &=\frac{1}{n^2} \Big(\sum_{i \neq j} \mathbb{E}(X_i)\mathbb{E}(X_j) + \sum_i \mathbb{E}(X_i^2)\Big)\\ &= \frac{1}{n^2} \Big( n(n-1)\mu^2 + n(\mu^2 + \sigma^2) \Big)\\ &= \frac{(n-1)\mu^2 + \mu^2 + \sigma^2}{n} =\mu^2 + \frac{\sigma^2}{n} \end{align*}

3.

\begin{align*} \mathbb{E} (X_j \bar{X})&= \frac{1}{n} \sum_i^n \mathbb{E}(X_jX_i) \\ &=\frac{1}{n} \big( \sum_{i\neq j}\mathbb{E}(X_j)\mathbb{E}(X_i) \big) + \frac{1}{n} \mathbb{E}(X_j^2)\\ &= \frac{1}{n} (n-1) \mu^2 + \frac{1}{n}(\mu^2 + \sigma^2)\\ &= \mu^2 + \frac{\sigma^2}{n} \end{align*} By using the values we have found we obtain

\begin{align*} \mathbb{E}(T_c) &= \frac{1}{c} \sum_{j=1}^n \mathbb{E}(X_j^2) + \mathbb{E}(\bar{X}^2) - 2 \mathbb{E} (X_j \bar{X})\\ &= \frac{1}{c} \sum_{j=1}^n (\sigma^2 + \mu^2) + ( \mu^2 + \frac{\sigma^2}{n} ) - 2 (\mu^2 + \frac{\sigma^2}{n}) \\ &= \frac{(n-1)\sigma^2}{c} \end{align*}

So $T_c$ is biased for $\sigma^2$ for all values of $c \neq n-1$

$\endgroup$
3
  • $\begingroup$ Thanks so much for the help!! I have a few questions though. For 2, I ended up trying something with Var($\bar X$) (similar to 1.) and ended up with the same answer as yours. Is it possible to use Var($\bar X$) = $\mathbb E ({\bar X}^2 )$ - $\mathbb E ( \bar X )^2$ ? For Var($\bar X$), i took out 1/n and it became 1/$n^2$ , which eventually gives me the $\sigma^2$ / n. In addition, I'm rather confused on your working for 2, I'm not exactly sure how you got from the 2nd to 3rd step. I'm quite confused on your working for 3 in general as well (from 1st to 2nd step, and first part of 3rd step) $\endgroup$ – Wei Xiong Yeo Sep 2 '18 at 16:18
  • $\begingroup$ Yes it is possible to use that formula and it is actually much easier to calculate that way :). Concerning how to get from 2nd to 3rd in 2 it's just explicitly multiplying all the terms in both sums. Concerning number 3: It comes down to calculating $\mathbb{E}(X_jX_i)$ in two different cases. If $i = j$ then $\mathbb{E}(X_jX_i) = \mathbb{E}(X_j^2)$. If $i \neq j$ then $X_j$ and $X_i$ are independent and we have $\mathbb{E}(X_jX_i) =\mathbb{E}(X_j)\mathbb{E}(X_i)$. $\endgroup$ – Digitalis Sep 2 '18 at 16:44
  • $\begingroup$ I don't know if I've ever before down-voted a clear and correct answer, but this seems to make the problem horribly more complicated than it really is. $\endgroup$ – Michael Hardy Sep 2 '18 at 20:00
1
$\begingroup$

$$ \frac 1 {\sigma^2} \sum_{i=1}^n (X_i-\overline X)^2 \sim \chi^2_{n-1}. $$ Therefore $$ \operatorname E\left( \frac 1 {\sigma^2} \sum_{i=1}^n (X_i - \overline X)^2 \right) = n-1. $$ So $$ \operatorname E\left( \frac 1 c \sum_{i=1}^n (X_i-\overline X)^2 \right) = \frac{\sigma^2} c (n-1). $$ Subtract $\sigma^2$ from that to get the bias.

$\endgroup$
8
  • $\begingroup$ Hi, I understand why the first expectation is n-1 (because of chi-squared), but how did you manage to get the second result without doing the lengthy working as mentioned above? Is it by manipulating the (n-1) result algebraically? So the first expectation found is solely used to find $\mathbb E \hat {\theta}$, and I just subtract $\theta$ which is $\sigma^2$ from it as per the bias formula? $\endgroup$ – Wei Xiong Yeo Sep 3 '18 at 2:05
  • 1
    $\begingroup$ @WeiXiongYeo : $$ \begin{align} & \text{We have } \operatorname E\left( \frac 1 {\sigma^2} \sum_{i=1}^n (X_i - \overline X)^2 \right) = n-1. \\ \\ & \text{Multiplying both sides by } \frac {\sigma^2} c, \text{ we get } \frac{\sigma^2} c \operatorname E\left( \frac 1 {\sigma^2} \sum_{i=1}^n (X_i-\overline X)^2\right) = \frac{\sigma^2} c (n-1). \\ \\ & \text{Using linearity of expectation we get } \operatorname E\left( \frac{\sigma^2} c \cdot \frac 1 {\sigma^2} \sum_{i=1}^n (X_i-\overline X)^2 \right) = \frac{\sigma^2} c (n-1). \end{align} $$ Then cancel $\sigma^2$ from the top and the bottom. $\endgroup$ – Michael Hardy Sep 3 '18 at 2:40
  • 1
    $\begingroup$ Thank you so much for the help too!! $\endgroup$ – Wei Xiong Yeo Sep 3 '18 at 3:22
  • 1
    $\begingroup$ $$ \begin{align} & \text{We have} \\ & \operatorname E\left( \frac 1 {\sigma^2} \sum_{i=1}^n (X_i - \overline X)^2 \right) = n-1. \\ \\ & \text{Multiplying both sides by } \frac {\sigma^2} c, \text{ we get} \\ \\ & \frac{\sigma^2} c \operatorname E\left( \frac 1 {\sigma^2} \sum_{i=1}^n (X_i-\overline X)^2\right) = \frac{\sigma^2} c (n-1). \\ \\ & \text{Using linearity of expectation we get} \\ \\ & \operatorname E\left( \frac{\sigma^2} c \cdot \frac 1 {\sigma^2} \sum_{i=1}^n (X_i-\overline X)^2 \right) = \frac{\sigma^2} c (n-1). \end{align} $$ $\endgroup$ – Michael Hardy Sep 5 '18 at 2:26
  • 1
    $\begingroup$ @WeiXiongYeo : You have $\operatorname{var}(\chi^2_k) = 2k.$ So the variance of the first expression in my answer is $2(n-1).$ If you multiply both sides by $\sigma^2/c,$ you multiply the variance by $\sigma^4/c^2,$ so you get $$ \operatorname{var}\left( \frac 1 c \sum_{i=1}^n (X_i - \overline X \right)^2 = \frac{\sigma^4}{c^2} 2(n-1). $$ $\endgroup$ – Michael Hardy Sep 5 '18 at 2:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.