# How to calculate the bias of the statistic

A given statistic : $T_c = \sum_{j=1}^n \frac{{(X_j - \bar X)}^2}{c}$, where $c$ is a constant, as an estimator of variance $\sigma^2.$

$X_1,\ldots, X_n$ denote a random sample from a population which has normal distribution with unknown mean $\mu$ and unknown variance $\sigma^2$.

The statistic is distributed as $x^2_{n-1}$ (a chi-squared variate with $n-1$ degrees of freedom).

I am tasked to find the bias of $T_c$.

I know the formula for bias is $\mathbb E \hat \theta - \theta$.

I found $\theta$ as $\mu = n - 1$ for a chi-squared distribution of $n-1$ degrees of freedom.

However, I am confused as to how to calculate $\mathbb E \hat \theta$.

What i thought of doing is to calculate $\mathbb E T_c$, but i got stuck halfway through, so I am not sure if I am doing the right thing. Any help please. thanks in advance.

• @StubbornAtom ah sorry just added the squared bracket for my $T_c$ – Wei Xiong Yeo Sep 1 '18 at 13:07
• Are you sure $T_c$ has distribution $\mathcal{X}^2_{n-1}$. Or did you mean to say that the $X_i$'s are $\mathcal{X}^2_{n-1}$ ? Also are the $X_i$'s independent ? – Digitalis Sep 1 '18 at 18:11
• @Digitalis Yes I would assume they are independent. Also, I have edited my original post to include the distribution of $X_i$'s – Wei Xiong Yeo Sep 2 '18 at 3:43

Since $T_c$ is an estimator of $\sigma^2$. $T_c$ will be unbiased for $\theta = \sigma^2$ if $\mathbb{E}_{\mu,\sigma^2}(T_c) = \sigma^2 \quad \forall \mu, \sigma^2 \in \mathbb{R} \times \mathbb{R}_0^+$

\begin{align*} \mathbb{E}(T_c)&= \frac{1}{c}\mathbb{E} \Big( \sum_{j=1}^n X_j^2 + \bar{X}^2 -2X_j \bar{X} \Big) \\ &= \frac{1}{c} \sum_{j=1}^n \mathbb{E}(X_j^2) + \mathbb{E}(\bar{X}^2) - 2 \mathbb{E} (X_j \bar{X}) \end{align*}

Now calculate $\mathbb{E}(X_j^2),\mathbb{E}(\bar{X}^2)$ & $\mathbb{E} (X_j \bar{X})$.

1. Using Var$(X) = \mathbb{E}(X^2) - \mathbb{E}(X)^2$ we find $\mathbb{E}(X_j^2) = \sigma^2 + \mu^2$.

2. \begin{align*} \mathbb{E}(\bar{X}^2) &= \mathbb{E}\Big( \big(\frac{1}{n} \sum_i^n X_i\big) \big(\frac{1}{n} \sum_i^n X_i\big)\Big) \\ &= \frac{1}{n^2} \mathbb{E}\Big( \big(\sum_i^n X_i\big) \big(\sum_i^n X_i\big)\Big)\\ &= \frac{1}{n^2} \mathbb{E}\Big( \sum_{i \neq j} X_iX_j + \sum_i X_i^2\Big) \\ &=\frac{1}{n^2} \Big(\sum_{i \neq j} \mathbb{E}(X_i)\mathbb{E}(X_j) + \sum_i \mathbb{E}(X_i^2)\Big)\\ &= \frac{1}{n^2} \Big( n(n-1)\mu^2 + n(\mu^2 + \sigma^2) \Big)\\ &= \frac{(n-1)\mu^2 + \mu^2 + \sigma^2}{n} =\mu^2 + \frac{\sigma^2}{n} \end{align*}

3.

\begin{align*} \mathbb{E} (X_j \bar{X})&= \frac{1}{n} \sum_i^n \mathbb{E}(X_jX_i) \\ &=\frac{1}{n} \big( \sum_{i\neq j}\mathbb{E}(X_j)\mathbb{E}(X_i) \big) + \frac{1}{n} \mathbb{E}(X_j^2)\\ &= \frac{1}{n} (n-1) \mu^2 + \frac{1}{n}(\mu^2 + \sigma^2)\\ &= \mu^2 + \frac{\sigma^2}{n} \end{align*} By using the values we have found we obtain

\begin{align*} \mathbb{E}(T_c) &= \frac{1}{c} \sum_{j=1}^n \mathbb{E}(X_j^2) + \mathbb{E}(\bar{X}^2) - 2 \mathbb{E} (X_j \bar{X})\\ &= \frac{1}{c} \sum_{j=1}^n (\sigma^2 + \mu^2) + ( \mu^2 + \frac{\sigma^2}{n} ) - 2 (\mu^2 + \frac{\sigma^2}{n}) \\ &= \frac{(n-1)\sigma^2}{c} \end{align*}

So $T_c$ is biased for $\sigma^2$ for all values of $c \neq n-1$

• Thanks so much for the help!! I have a few questions though. For 2, I ended up trying something with Var($\bar X$) (similar to 1.) and ended up with the same answer as yours. Is it possible to use Var($\bar X$) = $\mathbb E ({\bar X}^2 )$ - $\mathbb E ( \bar X )^2$ ? For Var($\bar X$), i took out 1/n and it became 1/$n^2$ , which eventually gives me the $\sigma^2$ / n. In addition, I'm rather confused on your working for 2, I'm not exactly sure how you got from the 2nd to 3rd step. I'm quite confused on your working for 3 in general as well (from 1st to 2nd step, and first part of 3rd step) – Wei Xiong Yeo Sep 2 '18 at 16:18
• Yes it is possible to use that formula and it is actually much easier to calculate that way :). Concerning how to get from 2nd to 3rd in 2 it's just explicitly multiplying all the terms in both sums. Concerning number 3: It comes down to calculating $\mathbb{E}(X_jX_i)$ in two different cases. If $i = j$ then $\mathbb{E}(X_jX_i) = \mathbb{E}(X_j^2)$. If $i \neq j$ then $X_j$ and $X_i$ are independent and we have $\mathbb{E}(X_jX_i) =\mathbb{E}(X_j)\mathbb{E}(X_i)$. – Digitalis Sep 2 '18 at 16:44
• I don't know if I've ever before down-voted a clear and correct answer, but this seems to make the problem horribly more complicated than it really is. – Michael Hardy Sep 2 '18 at 20:00

$$\frac 1 {\sigma^2} \sum_{i=1}^n (X_i-\overline X)^2 \sim \chi^2_{n-1}.$$ Therefore $$\operatorname E\left( \frac 1 {\sigma^2} \sum_{i=1}^n (X_i - \overline X)^2 \right) = n-1.$$ So $$\operatorname E\left( \frac 1 c \sum_{i=1}^n (X_i-\overline X)^2 \right) = \frac{\sigma^2} c (n-1).$$ Subtract $\sigma^2$ from that to get the bias.

• Hi, I understand why the first expectation is n-1 (because of chi-squared), but how did you manage to get the second result without doing the lengthy working as mentioned above? Is it by manipulating the (n-1) result algebraically? So the first expectation found is solely used to find $\mathbb E \hat {\theta}$, and I just subtract $\theta$ which is $\sigma^2$ from it as per the bias formula? – Wei Xiong Yeo Sep 3 '18 at 2:05
• @WeiXiongYeo : \begin{align} & \text{We have } \operatorname E\left( \frac 1 {\sigma^2} \sum_{i=1}^n (X_i - \overline X)^2 \right) = n-1. \\ \\ & \text{Multiplying both sides by } \frac {\sigma^2} c, \text{ we get } \frac{\sigma^2} c \operatorname E\left( \frac 1 {\sigma^2} \sum_{i=1}^n (X_i-\overline X)^2\right) = \frac{\sigma^2} c (n-1). \\ \\ & \text{Using linearity of expectation we get } \operatorname E\left( \frac{\sigma^2} c \cdot \frac 1 {\sigma^2} \sum_{i=1}^n (X_i-\overline X)^2 \right) = \frac{\sigma^2} c (n-1). \end{align} Then cancel $\sigma^2$ from the top and the bottom. – Michael Hardy Sep 3 '18 at 2:40
• Thank you so much for the help too!! – Wei Xiong Yeo Sep 3 '18 at 3:22
• \begin{align} & \text{We have} \\ & \operatorname E\left( \frac 1 {\sigma^2} \sum_{i=1}^n (X_i - \overline X)^2 \right) = n-1. \\ \\ & \text{Multiplying both sides by } \frac {\sigma^2} c, \text{ we get} \\ \\ & \frac{\sigma^2} c \operatorname E\left( \frac 1 {\sigma^2} \sum_{i=1}^n (X_i-\overline X)^2\right) = \frac{\sigma^2} c (n-1). \\ \\ & \text{Using linearity of expectation we get} \\ \\ & \operatorname E\left( \frac{\sigma^2} c \cdot \frac 1 {\sigma^2} \sum_{i=1}^n (X_i-\overline X)^2 \right) = \frac{\sigma^2} c (n-1). \end{align} – Michael Hardy Sep 5 '18 at 2:26
• @WeiXiongYeo : You have $\operatorname{var}(\chi^2_k) = 2k.$ So the variance of the first expression in my answer is $2(n-1).$ If you multiply both sides by $\sigma^2/c,$ you multiply the variance by $\sigma^4/c^2,$ so you get $$\operatorname{var}\left( \frac 1 c \sum_{i=1}^n (X_i - \overline X \right)^2 = \frac{\sigma^4}{c^2} 2(n-1).$$ – Michael Hardy Sep 5 '18 at 2:36