2
$\begingroup$

So I generally understand the basis for the t-test: i.e. you take advantage of the fact that you can make $\bar{X}-\bar{Y}$ standard normal:
$$Z = \frac{((\bar{X}-\bar{Y})-(\mu_X-\mu_Y))}{\sigma \sqrt{\frac{1}{n}+\frac{1}{m}}}$$
where $\sigma$ is the variance shared by the normal random variables $X$ and $Y$.
Furthermore, the unbiased estimator for $\sigma^2$ is:
$$S_p^2 = \frac{1}{m+n-2}\left(\sum{(X_i-\bar{X})^2}+\sum{(Y_i-\bar{Y})^2}\right)$$
which can be represented as a $\chi^2$ random variable using the fact that:

$\;\;\;\;\;\;\;\;\: \dfrac{(m+n-2)S_p^2}{\sigma^2}\:\:$ is $\:\:\:\chi^2$

and then you combine these distribution functions to get a T distribution.

What I don't understand is why the variances have to be equal (in this case, not talking about the wilcox test or anything like that). For example, what if you just had $\sigma_X^2 = a \sigma_Y^2$? where you knew the constant $a$. I tried to go through the derivation for the T-distribution and didn't come across any problems, I just had to include the constant $a$ in different places. If someone could direct me towards a reference or help me understand why this is the case I'd be very appreciative. I tried searching for the answer with no luck.

Thanks!

$\endgroup$
  • 1
    $\begingroup$ Then you should go for $t$ test with unequal variances. $\endgroup$ – Jimmy R. Apr 2 '16 at 16:04
  • $\begingroup$ I'm more looking for an explanation, not really interested in application at this point $\endgroup$ – lstbl Apr 2 '16 at 16:08
  • 1
    $\begingroup$ I think you are assuming $\alpha$ in $\sigma_X^2 = \alpha \sigma_Y^2$ is known. Then you still really have only one unknown variance. The trouble is trying to get the math to go through when there are two different unknown variances. $\endgroup$ – BruceET Apr 5 '16 at 4:48
3
$\begingroup$

If the population variances are not known to be equal, then the appropriate statistic for testing equality of population means or finding a confidence interval for their difference would be (in your notation)

$$T = \frac{(\bar X = \bar Y) - (\mu_X - \mu_Y)} {\sqrt{\frac{S_X^2}{n}+\frac{S_Y^2}{m}}}.$$

Distribution theory. But attempts do derive the exact distribution of this statistic in a useful form have been unsuccessful. This is known as the 'Behrens-Fisher Problem'. Nevertheless, it has been shown that the distribution is approximately Student's t with degrees of freedom between $\min(n-1, m-1)$ and $n + m - 2.$ The precise formula for df is given in many statistics texts, but messier than I want to type here. (Google 'Welch-Satterthwaite equation'.)

Comprehensive simulation studies have shown that this approximate t distribution is very accurate for a wide variety of ratios $\sigma_X^2/\sigma_Y^2$ and choices of $m$ and $n$.

In practice. A consensus seems to have developed among applied statisticians that it is best to use 'Welch' or 'separate variances' t procedures, instead of 'pooled t' procedures, in all cases---regardless whether an F-test shows that $\sigma_X^2 \ne \sigma_Y^2.$ Many software packages use the Welch test as their default two-sample t test, doing the pooled test only if one overrides the default. (Two examples in my experience are R and Minitab.)

The pooled test has the advantage of being workable on a statistics exam where calculators are available, but not statistical software. This may account for its continued use in the classroom. Even so, many recent elementary texts (including the popular elementary books by Moore) recommend using the Welch test with $df = \min(n-1, m-1).$ This is a conservative choice (never rejecting when the messier, more accurate df would fail to reject). And if both sample sizes are above 30 or so, this is not much of a compromise.

Example. Here is an example using fake normal data to illustrate output of the Welch test in R. Sample 1 has 10 observations from $Norm(\mu=100, \sigma=15)$ and Sample 2 has 13 observations from $Norm(\mu=80, \sigma=20).$

Because the population means differ we might hope for rejection, but the P-value is a little above 5% for the particular sample generated. The population standard deviations (hence variances) are large and unequal. (Of course, other samples generated using the same parameters might show different test outcomes.)

In particular, notice that $df \approx 18$ here, whereas a pooled t test would have $df = 10 + 13 - 2 = 21,$ Roughly, the df for the Welch test is decreased below $n + m - 2$ more markedly as the two sample variances differ more markedly.

 x1 = round(rnorm(10, 100, 15), 1);  sort(x1)
 ## 83.2  85.9  88.3  92.3  92.4  92.5  96.3 101.4 104.6 124.2
 x2 = round(rnorm(13, 80, 20), 1);  sort(x2)
 ##  47.7  51.1  56.2  61.3  65.8  66.6  81.9  83.5  88.1  97.8 100.6 104.2 136.0

 t.test(x1, x2, alte="two.sided")

 ##        Welch Two Sample t-test

 ## data:  x1 and x2 
 ## t = 2.0137, df = 17.854, p-value = 0.05936
 ## alternative hypothesis: true difference in means is not equal to 0 
 ## 95 percent confidence interval:
 ## -0.7048102 32.8017333 
 ## sample estimates:
 ## mean of x mean of y 
 ## 96.11000  80.06154 

 Values = c(x1, x2);  Group = c(rep(1,10),rep(2,13))
 stripchart(Values ~ Group, ylim=c(.5, 2.5))

enter image description here

Reference: The Wikipedia article on 'Welch t test' gives formulas and additional examples. Unfortunately, it uses the phrase 'unequal variances test' where I would prefer 'separate variances test' because population variances need not be unequal in order for the test to be valid.

$\endgroup$
  • $\begingroup$ This is a great, practical explanation of why the Welch T-test is a preferred test. Thanks Bruce. $\endgroup$ – lstbl Apr 6 '16 at 16:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.