0
$\begingroup$

I have a question regarding one formula (3.8) on page 84 of the sixth edition of financial risk manager handbook. For convenience, I will rephrase the problem as below.

Define $X$ as the random variable of interest. We observe a sequence of $T$ realized values for $x$, $x_{1},x_{2},\ldots, x_{T}$. The sample mean is \begin{equation*} m=\hat\mu = \frac{1}{T}\sum_{i=1}^{T}x_{i} \end{equation*} The sample variance is defined as \begin{equation*} s^{2}=\hat\sigma^{2} = \frac{1}{T-1}\sum_{i=1}^{T}(x_{i}-\hat\mu)^{2} \end{equation*} Suppose $x_{1},x_{2},\ldots, x_{T}$ have a normal distribution with mean $\mu$ and variance $\sigma^{2}$, it has been proved that \begin{equation*} (T-1)\hat\sigma^{2}/\sigma^{2} \sim \chi^{2}(T-1) \end{equation*} It has also been proved that if the sample size $T$ is large enough, the chi-square distribution converges to a normal distribution, then: \begin{equation} \hat\sigma^{2} \sim N(\sigma^{2},\sigma^{4}\frac{2}{T-1}) \end{equation} The central limit theorem states that this distribution is only valid asymptotically. How come the result that, approximately, the sample standard deviation $\hat\sigma$ has a normal distribution with a standard error of \begin{equation} se(\hat\sigma)=\sigma \sqrt {\frac{1}{2T}} \end{equation} ? By definition of standard error, $se(\hat\sigma)=\sqrt{E[\hat \sigma^{2}-E(\hat\sigma^{2})]^{2}}=\sqrt{E[\hat\sigma^{2}]-(E[\hat\sigma])^2}=0$ I am wondering how to get the standard error $\sigma\sqrt{\frac{1}{2T}}$? Thanks!

$\endgroup$
  • $\begingroup$ No, $\sqrt{E[\hat\sigma^{2}]-(E[\hat\sigma])^2}=0$ doesn't hold, why should it ? And by the way, any random variable (including $\hat\sigma$) has a nonzero standard deviation. $\endgroup$ – Yves Daoust Jun 16 '17 at 20:07
  • $\begingroup$ I should correct it as $se (\hat \sigma) = \sqrt{E[\hat \sigma -E[\hat \sigma ]]^2} = \sqrt{E[\hat \sigma ^2] -(E[\hat \sigma ])^2} $ which is not zero. $\endgroup$ – cmd1991 Jun 18 '17 at 8:46
0
$\begingroup$

I think you may be confusing two kinds of estimation:

$1.$ For estimating $\mu,$ the estimator is $\bar X = \frac{1}{n}\sum_i X_i.$ Then the standard error (SE) of $\bar X$ is $SD(\bar X) = \sigma/\sqrt{n}.$ When $\sigma$ is unknown, estimated by $S$, the (estimated) SD is $S/\sqrt{n}.$ Because the case with $\sigma$ unknown is the most common in practice, the word estimated is often dropped out of laziness. (I have used $n$ instead of your $T$ in order to avoid confusion with notation in the next paragraph.)

To make a 95% CI for $\mu$, the usual procedure is to observe that the statistic $T = \frac{\bar X - \mu}{S/\sqrt{n}} \sim \mathsf{T}(df=n-1),$ so that one can find constants $L$ and $U$ with $P(L < T < U) = 0.95.$ With simple manipulation of inequalities, one finds a CI of the form $\bar X \pm t^*S/\sqrt{n},$ where it is customary to use $t^* = -L = U$ by symmetry, where $t^*$ cuts 2.5% from each tail of $\mathsf{T}(df=n-1).$ Roughly speaking, one says that the CI is the point estimate $\bar X$ of $\mu$ plus or minus a 'probability factor' times the standard error.

$2.$ For estimation $\sigma^2$, as you say, one uses the fact that $Q = \frac{(n-1)S^2}{\sigma^2} \sim \mathsf{Chisq}(df = n-1).$ Thus to make a 95% CI for $\sigma,$ one can find $L$ and $U$ (different from above) so that $P(L < Q < U) = 0.95$ and the CI is of the form $\left(\frac{(n-1)S^2}{U}, \frac{(n-1)S^2}{L}\right).$ Because the chi-squared distribution is not symmetrical, $L>0$ and $U>L$ have different numerical values. Often one chooses $L$ to cut 2.5% of the probability from the lower tail of $\mathsf{Chisq}(df = n-1)$ and $U$ to cut 2.5% from its upper tail. (This is sometimes called a 'probability-symmetric' CI; generally, not the shortest possible CI, but a convenient choice. (To find a CI for $\sigma,$ take square roots of the endpoints of the CI for $\sigma^2.$)

In practical applications, one does not often speak of the SE of the estimator $S^2$ of $\sigma^2$ or $S$ of $\sigma$ because SE does not play an explicit role in finding confidence limits. Because $S^2$ is an unbiased MLE of $\sigma^2$ and the usual "regularity conditions" are satisfied, $S^2$ has the smallest variance among unbiased estimators of $\sigma^2$ and that implies that CIs based on $S^2$ tend to be the 'best' ones available, and applied discussions often end there.

It is true that $S^2 = \widehat{\sigma^2}$ is asymptotically normal, so that for very large $n$, CIs for $\sigma^2$ become nearly symmetrical, but the form of CI I showed above is the one in standard use.

Addendum (per Comments): The terminology 'standard error' is typically used to mean the standard deviation of an estimator. I am not sure what you are estimating. For a sample of size $n$ from $\mathsf{Norm}(\mu, \sigma),$ one can manipulate the gamma distribution of $S_n^2$ to show that $E(S_n) = \sigma\sqrt{\frac{2}{n-1}}\Gamma\left(\frac{n}{2}\right)/\Gamma\left(\frac{n-1}{2}\right).$ Expressing the $\Gamma$-function in terms of factorials by Stirling's approximation, one can show that $E(S_n) \rightarrow \sigma.$ I suppose you could use $E(S_n)$ to find $Var(S_n)$ (since you know $E(S_n^2)$) and hence $SD(S_n),$ if that is what you want. Frankly, I'm not sure whether you mean $SE(\hat \sigma)$ to be $SD(S_n)$ or whether either is approximately equal to $\sigma\sqrt{.5/n}.$

A simulation in R statistical software of a million sample standard deviations $S_n$ each with $n = 50$ and $\sigma = 1$ shows $SD(S_n) \approx \sigma\sqrt{\frac{1}{2n}} = 0.1,$ which might be promising.

m = 10^6;  n = 50
s = replicate(m, sd(rnorm(n)))
sd(s)
## 0.100835  # aprx 0.1 as anticipated

However, a histogram of simulated values $S_n^2$ is not consistent with the density function of $\mathsf{Norm}(1, \sqrt{2/49})$ (second argument is SD). So $n = 50$ is not nearly a large enough sample size for asymptotic normality of $S^2.$

enter image description here

Perhaps there are enough clues between these comments and those of @Yves (+1) for you or someone else to resolve this to your satisfaction.

Also, some of this is discussed in this link.

$\endgroup$
  • $\begingroup$ Thanks for your explanation! I understand that you mean that $se(\hat \sigma)$ is not so important because it won't be used in finding CIs. However, I am still wondering how to derive $se(\hat \sigma) =\sigma \sqrt{\frac{1}{2T}}$ from the distribution of $\hat \sigma ^2$. Or, could you explain a little more about the relationship between $se(\hat \sigma ^2)$ and $se(\hat \sigma)$? $\endgroup$ – cmd1991 Jun 18 '17 at 8:29
  • $\begingroup$ See the Addendum to my Answer (and the link at the end and the link there). $\endgroup$ – BruceET Jun 18 '17 at 22:38
  • 1
    $\begingroup$ Thanks. Perhaps you would find this link which includes mathematical analysis useful. Also, this article gives good explanation. $\endgroup$ – cmd1991 Jun 19 '17 at 1:12
1
$\begingroup$

For $x_i$ normally distributed, there is no need to invoke the central limit theorem.

The statistical distribution of the estimator of the variance $\hat\sigma^2$ is indeed a $\chi^2$ law with $T-1$ degrees of freedom (to a factor $\sigma^2/(T-1)$), i.e. a sum of squared normal variables. Then the expectation of this estimator is is computed exactly from the $\chi^2$ distribution as the unbiaised value

$$\mu_{\hat\sigma^2}=\sigma^2$$

and the variance,

$$\sigma^2_{\hat\sigma^2}=\frac{2\sigma^4}{T-1}.$$

$\endgroup$
  • 1
    $\begingroup$ Agee with you that there is no need to invoke CLT to obtain the mean and variance of $\hat \sigma ^2$. However, I am still confused about how to get the approximation $se(\hat \sigma) = \sigma \sqrt{\frac{1}{2T}}$ from the distribution of $\hat \sigma ^2$. $\endgroup$ – cmd1991 Jun 18 '17 at 8:17
  • $\begingroup$ @cmd1991: where did you get this from ? $\endgroup$ – Yves Daoust Jun 18 '17 at 13:38
  • $\begingroup$ It comes from the FRM material "The Financial Risk Manager Handbook Plus Test Bank" by Philippe Jorion, 2010. On page 64 of the sixth edition, formula (3,8) is what I would like to figure out how to derive. $\endgroup$ – cmd1991 Jun 18 '17 at 15:19
  • 1
    $\begingroup$ @cmd1991 The standard deviation of the variance estimator is the square root of the variance of the variance estimator, $\sigma^2\sqrt{2/(T-1)}$. But the standard deviation of the standard deviation estimator is different and pretty hard to evaluate analytically as the latter follows the square root of a $\chi^2$. $\endgroup$ – Yves Daoust Jun 18 '17 at 15:28
  • $\begingroup$ Got you. I also find it hard to evaluate so I was confused by the text book saying "By the same approximation, the sample standard deviation has a normal distribution with a standard error of $se(\hat \sigma) = \sigma \sqrt{\frac{1}{2T}}$, which seems like the result is natural. Still, thank you very much. $\endgroup$ – cmd1991 Jun 18 '17 at 15:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.