12
$\begingroup$

(note: this is a very basic probability question, so it is highly probable (heh) that it is a duplicate)

Every time I am trying to get into statistics (again), I am always lost at hypothesis testing.

My basic question is - why do we form a null hypothesis as a negation of what we want to prove in the first place, and only then do we prove or disprove the null hypothesis?

Why do we do it at all, instead of just proving the original hypothesis?

$\endgroup$
5
  • 5
    $\begingroup$ You never prove a null hypothesis statistically. $\endgroup$ Dec 14, 2011 at 2:02
  • $\begingroup$ Brian: why not, if you got data, that support it with some confidence? can't you say you proved the null hypothesis and thus, disproved your original hypothesis (which is now called "alternative hypothesis") with that same confidence? $\endgroup$ Dec 14, 2011 at 2:07
  • $\begingroup$ OK, I am maybe confused what "rejecting" means, then. I was under the impression that "rejecting" and "accepting" are logical opposites. In my mental model, if you reject A, you therefore accept (not A). And vice versa. $\endgroup$ Dec 14, 2011 at 2:17
  • 7
    $\begingroup$ Related: here and here and here and here and ... $\endgroup$
    – cardinal
    Dec 14, 2011 at 2:36
  • 7
    $\begingroup$ ...here and here and here and here. $\endgroup$
    – cardinal
    Dec 14, 2011 at 2:36

5 Answers 5

19
$\begingroup$

You seem confused. Statistical methods do not set out to prove; one either rejects or fails to reject a hypothesis. This wording is Very Important™. (See also this and this.)

I'll use the classical judicial analogy. The accused (hypothesis) standing before the judge can be taken as "guilty" or "not guilty" by the judge (hypothesis test). Even with this, we can't totally eliminate the possibility of committing a Type I (innocent goes to jail) or Type II (guilty goes free) error. For all we know, even with all the evidence considered by the prosecution, defense, and jury, there might be a few confounding factors that weren't seen at the time. (Think of all the cases whose verdicts got changed when DNA tests became vogue.)

Put another way, using the word "accept" misleads some people. Here, it means that we're accepting the possibility that it's true, not that it is certainly true.

$\endgroup$
10
  • 6
    $\begingroup$ In fact, the typical results in the American justice system are "guilty" and "not guilty" (not "guilty" vs. "innocent"). A finding of "not guilty" does not correspond directly to innocence. It corresponds to "not enough evidence to find the defendant guilty". So, someone may actually be guilty of a crime but there may be insufficient evidence to convict them as such. In statistical terms, this would correspond to failing to reject the null hypothesis of innocence. $\endgroup$
    – mhum
    Dec 14, 2011 at 2:30
  • 2
    $\begingroup$ @Karel: that's a good summary. It controls the type I error and then if the type II error suffers this is called low power. But when you say "costly" this is getting dangerously close to talking about loss functions. $\endgroup$
    – opt
    Dec 14, 2011 at 3:25
  • 3
    $\begingroup$ A-ha! So, we seek out to reject the null hypothesis with a given certainty, because we care much more about not saying something incorrect rather than about not saying something correct. OK. For me, the key to (at least partially) understanding the null hypothesis testing are the two types of the errors. $\endgroup$ Dec 14, 2011 at 3:43
  • 3
    $\begingroup$ Great answer. If only this focus on what statistics cannot do was more prevalent... $\endgroup$
    – Did
    Dec 14, 2011 at 8:04
  • 2
    $\begingroup$ @Didier: coming from you, this means a lot. Thanks for the nice comment! $\endgroup$ Dec 14, 2011 at 8:07
6
$\begingroup$

I realise this has already been answered very well, but actually the point you raise, whether out of confusion or not, is very valid. In fact there has been controversy in the past raised by the "obsessive" focus on the rejection of the null hypothesis.

For example, in "The Fallacy of the Null-Hypothesis Significance Test" by Rozeboom (1960), the following conclusion is drawn:

The traditional null-hypothesis significance-test method ... is here vigorously excoriated for its inappropriateness as a method of inference. While a number of serious objections to the method are raised, its most basic error lies in mistaking the aim of a scientific investigation to be a decision, rather than a cognitive evaluation of propositions. It is further argued that the proper application of statistics to scientific inference is irrevocably committed to extensive consideration of inverse probabilities, and to further this end, certain suggestions are offered, both for the development of statistical theory and for more illuminating application of statistical analysis to empirical data.

Furthermore, in "Consequences of Prejudice Against the Null Hypothesis" by Greenwald (1975), the following conclusion is given,

Accordingly, it is concluded that research traditions and customs of discrimination against accepting the null hypothesis may be very detrimental to research progress.

More recently an alternative method was given in "An Alternative to Null-Hypothesis Significance Tests" (Killeen 2005)

The statistic $P_{rep}$ estimates the probability of replicating an effect. It captures traditional publication criteria for signal-to-noise ratio, while avoiding parametric inference and the resulting Bayesian dilemma. In concert with effect size and replication intervals, $P_{rep}$ provides all of the information now used in evaluating research, while avoiding many of the pitfalls of traditional statistical inference.

$\endgroup$
2
3
$\begingroup$

A very basic explanation is that having only one hypothesis around will tell you nothing. You can get pretty numbers out of statistical analysis, but without something to compare them to, you'll be none the wiser.

As soon as you have two (or more) hypotheses to pit against each other, the analysis can begin to tell you something about how large a leap of faith it will require to conclude that the observed data are caused by this hypothesis rather than that. Thus, the null hypothesis: something neutral and boring that you can compare other hypotheses to.

In order to find out which answer is the best, you need to have more than one possible answer you consider. Otherwise you can't even speak of whether this answer is better or worse than that one.

$\endgroup$
1
$\begingroup$

The part where you get lost is probably where you are thinking in Bayesian terms but you are trying to learn frequentist methods.

$\endgroup$
3
  • $\begingroup$ Please, elaborate more. Yes, I am using Bayesian probability all the time in my work and the rest of probability theory only time from time. $\endgroup$ Dec 14, 2011 at 1:59
  • 8
    $\begingroup$ Bayesian vs. frequentist is a thing, and the difference is causing your confusion. Your intuition is Bayesian, whereas the technique you are learning is frequentist. The internet probably has as much Bayesian vs. frequentist literature as emacs vs. vi so it shouldn't be hard to find. It might not answer all of your questions but at least it should help to clarify the distinction. $\endgroup$
    – opt
    Dec 14, 2011 at 2:15
  • 2
    $\begingroup$ "as much Bayesian vs. frequentist literature as emacs vs. vi" - I like this one. :D $\endgroup$ Dec 14, 2011 at 3:04
1
$\begingroup$

To prove the original hypothesis, you want to be sure the results have been obtained because of your alternative hypothesis, rather than by chance. The burden of proof has to be on the alternative hypothesis. If I believe fairies are real, but you cannot see them, then this is consistent with the world we see. The null hypothesis is that they are not real. I need to therefore come up with some seriously good evidence to reject that null hypothesis, before we can scientifically say that fairies are probably real.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .