Statistical Hypothesis Testing

Question

Does stress have an influence on the time people need to finish a task? To answer this question 2 groups of test people are compared. One of the groups ($n_1=10)$ does a task under the influence of stress, and the other group $(n_2=12)$ does the same task without stress. The average times (in seconds): $\bar{x}_1 =433$ and $\bar{x}_2 = 367$. The sample standard deviations are: $s_1 = 65$ and $s_2 = 84$.

I need to test this hypothesis. I'm however having trouble with this because I never really learned how to test hypotheses using statistics, and I don't know any 'plan of attack' as to how to tackle hypothesis testing. This is what I have up until now:

$X$ = time in secs

Group 1 (with stress):

$n_1 = 10$

$\mu_{\bar{x}_1} = 433$

$ \sigma_{\bar{x}_1} = 65$

Group 2 (without stress):

$n_2 = 12$

$ \mu_{\bar{x}_2} = 367$

$ \sigma_{\bar{x}_2} = 84$

But that's about it. The null hypothesis is that stress doesn't have an effect, but I don't know how to describe that mathematically. So I'm stuck. Is there a general way to do hypothesis testing? What is my missing link in this specific case?

Answer 1

There is a "general way" to do hypothesis testing (more than one, in fact), but that's a topic for a whole Mathematical Statistics course rather than a Math.SE question. Most people who do statistical hypothesis tests never learn about it; it's sort of behind the scenes.

In this setting, where you want to test the equality of two group means for populations which are something like normally distributed, what they want you to do is a t-test. Since we have two separate samples (Group 1 and Group 2), this is a two-sample t-test. Since the people in the groups are different (person 1 in group 1 doesn't correspond in any meaningful way to person 1 in group 2) it's a non-paired two-sample t-test. (Paired two-sample t-tests include settings such as twin studies where one twin is in each group and before-after measurements).

Since the standard deviations of the two groups are not hugely different, it's probably safe to use the pooled t-test. (If you want to be a bit more conservative you can do the Welch-Satterthwaite t-test). The pooled standard deviation is calculated as $$s_{p} = \sqrt{\frac{(n_1-1)s_{1}^2+(n_2-1)s_{2}^2}{n_1+n_2-2}}.$$ The test statistic is given by $$t = \frac{\bar {x}_1 - \bar{x}_2}{s_{p} \cdot \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}.$$ The test statistic is compared to a $t$ distribution with $n_1 + n_2 − 2$ degrees of freedom to find the $p$-value. You reject the null hypothesis if the $p$-value is smaller than some cut-off $\alpha$ (often .05).

Answer 2

The rationale for the T-test (testing the significance of the difference of the means in two subgroups) is, that if you take infinitely many samples of size N, then the empirical means are also "normally" distributed around the true population mean. This makes, that you can define a "confidence interval" around your one sample mean, in which you can expect the true population mean (with a certain (small, say 5%) risk of error).
If you have two subgroup means in your sample, then each of them can be seen as base for such a prognosis of the true mean in the population.
If now the confidence interval for each erstimated population mean is small, and the confidence intervals around each subgroup mean do not overlap much, then you can be confident (with error risk) that also in the population each group has its own mean (say, body heights of men vs women).
The way to determine the confidence intervals is simple, as long as we can assume that the items are "normal distributed".
Then you compute the SEMean (the standard error for the sample means) by $$ \operatorname{semean}_1= {\operatorname{stddev}_1 \over \sqrt{N_1} } $$ and for the second group $$ \operatorname{semean}_2={ \operatorname{stddev}_2 \over \sqrt{N_2} } $$. Then the confidence interval, such that with 95% likelihood the true population mean is inside the interval around the empirical mean for the subgroup in your sample, is determined by $$ \operatorname{lowerbound}_{1,95 \% } = \operatorname{mean}_1 - 2 \operatorname{semean}_1 \\ \operatorname{upperbound}_{1,95 \% } = \operatorname{mean}_1 + 2 \operatorname{semean}_1 $$ You do the same for the mean and the confidence interval of the second subgroup.
You have a rough answer if the confidence intervals do not overlap (or not significantly overlap): then it is very likely, that indeed each group has also its own mean in the population. This is the general rationale for this type of questions.
Just try this with your data for a first impression, "how distinct" are actually the means under stress and without stress.
The t-test does now the precise computation of a numerical value for the significane/likelihood, that those estimated means in the population are distinct. Here some more precision is involved: are the subgrpoups of same size? Are the deviances in the subgroups comparable? (See for this the other anser above). But again: as we assume normal distribution in our items, this can be done with the -relatively easy- formulae for the likelihoods and cumulated likelyhoods of the normal distribution.