Let $H_0:\mu=\mu_0$ versus $H_1:\mu>\mu_0$ be statistical hypothesis. First a definition: What is the probability of observing a data which is similar to the one on hand or more extreme if $H_0$ happens to be true? This probability will be called the $p$-value.

Now what is meant by more extreme here: $\bar{X}>\bar{x}$ or $\bar{X}<\bar{x}$ and why? Only some intuition is required.

It is unclear to me how do we use that $H_0$ is true when we write $\bar{X}>\bar{x}$ or $\bar{X}<\bar{x}$.

  • $\begingroup$ Not sure what distinction you're making between $\bar X$ and $\bar x$. $\endgroup$ – BruceET Feb 15 '18 at 3:26

When performing a hypothesis test, we have a null hypothesis and an alternative hypothesis. To go with this, we also have a null distribution, that is the distribution assuming that the null hypothesis is true (true mean is $\mu_0$), here we assume that the population variance is known, $\sigma^2$.

Let's assume we are working with a normal distributed population, in this case then the null distribution is bell-shaped and centered at $\mu_0$. Then we observe $n$ samples from our data, and we find the sample average $\bar{X}$. We want to see how bad of an assumption that $\mu = \mu_0$, in other words, we want to see how far away is what we observe from what we would expect to observe.

We standardise the sample mean, purely for convenience. Under the null hypothesis, the population is gaussian with mean $\mu_0$, and variance $\sigma^2$. So that means the sample mean has distribution: $\bar{X} \sim N(\mu_0, \frac{\sigma^2}{n})$. So, this can be standardised by:

$$ Z =\frac{\bar{X} - \mu_0}{\sigma / \sqrt{n}} $$

which should follow a standard normal $N(0,1)$ (if the null is true).

We know that for a standard gaussian, approximately:

  • 67% of the values lie within 1 standard deviation of the mean
  • 95% within 2 standard deviations
  • 99% within 3 standard deviations

For the standard gaussian, the standard deviation is $\sigma=1$, so 99% of the time, we would expect samples from this distribution to fall within the range $[-3,3]$ (actually it's more like $\pm 2.96$.

So, for example, if our statistic $Z = 1.2$, then it doesn't seem to be too extreme an assumption that the population mean is in fact $\mu_0$. However, if $Z=4.5$, this seems very unlikely, and indicates that our null distribution is most likely wrong.

The $p$-value simply measures this probability. From wiki, the p-value is the probability (under the null hypothesis) of obtaining a result equal to or more extreme than what was actually observed. In the case $Z=1.5$, we would have a high $p-value$, but for $Z=4.5$, we would have a very small p-value as $4.5$ falls very far out.

How extreme is extreme: this is chosen by the individual performing the test, you might think that being within $/pm 2$ standard deviations is tolerable, or you might put the cut off at $\pm 2.5$ standard deviations. Depending on what you choose, you will have a definition of extreme to go along

update We chose our null hypothesis as $H_0: \mu = \mu_0$, and when creating a test we need to choose a significance level. The significance level is the probability of a type 1 error (rejecting the null hypothesis when it is true). Let's take the significance level to be $\alpha \in [0,1]$, in practice, $\alpha$ tends to be one of 0.01, 0.05, 0.1.

Now, if our alternative hypothesis is: $H_1: \mu > \mu_0$, then we have a one sided test. We look at the farthest $\alpha$% of our standard normal distribution to the right, and call this our critical region. If $\alpha = 0.05$, our critical region would be the interval $[1.65, \infty)$. If we observe a value within the interval, we would reject it (for example, Z=1.8). Similarly, if we had $H_1: \mu < \mu_0$, we would reject values in the interval: $(-\infty, -1.65 ]$, the most extreme $5$% to the left of the null distribution.

Finally, if our alternative hypothesis was $H_1: \mu \neq \mu_0$, we now need to consider both left and right extremes of the null. With confidence level $\alpha$, we have critical regions: $(-\infty, \alpha/2] \cup [\alpha/2, \infty)$

  • $\begingroup$ I have some comments about your nice and long answer. First, how/where did you use $H_1:\mu>\mu_0$? Second your displayed $Z$ should have square root in the denominator. And finally, what is "$/pm2$"? $\endgroup$ – user122424 Feb 15 '18 at 14:33
  • $\begingroup$ If $H_0:\mu=\mu_0$ and the alternative hypothesis is $H_1:\mu<\mu_0$ and $Z=4.5$ then we accept $H_0$ though it has a very small $p-value$, is that right? $\endgroup$ – user122424 Feb 15 '18 at 15:28
  • $\begingroup$ @user122424 just picture a standard normal, with mean 0 and variance 1, we expect 99% of the values to fall within $\pm 3$. Then we see a value of 4.5, this is very far out to the right, the probability of observing 4.5 or greater, ie, $p(Z > 4.5) \approx 0$, which is our p-value, we would definitely not accept this $\endgroup$ – dimebucker Feb 15 '18 at 23:04
  • $\begingroup$ @user122424 does the update clarify things for you? $\endgroup$ – dimebucker Feb 19 '18 at 0:12
  • $\begingroup$ Yes, it does. But I may well ask you in the near future, when I reread it more times. $\endgroup$ – user122424 Feb 20 '18 at 18:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.