I realise this has already been answered very well, but actually the point you raise, whether out of confusion or not, is very valid. In fact there has been controversy in the past raised by the "obsessive" focus on the rejection of the null hypothesis.
For example, in "The Fallacy of the Null-Hypothesis Significance Test" by Rozeboom (1960), the following conclusion is drawn:
The traditional null-hypothesis significance-test method ... is here vigorously excoriated for its inappropriateness as a method of inference. While a number of serious objections to the method are raised, its most basic error lies in mistaking the aim of a scientific investigation to be a decision, rather than a cognitive evaluation of propositions. It is further argued that the proper application of statistics to scientific inference is irrevocably committed to extensive consideration of inverse probabilities, and to further this end, certain suggestions are offered, both for the development of statistical theory and for more illuminating application of statistical analysis to empirical data.
Furthermore, in "Consequences of Prejudice Against the Null Hypothesis" by Greenwald (1975), the following conclusion is given,
Accordingly, it is concluded that research traditions and customs of discrimination against accepting the null hypothesis may be very detrimental to research progress.
More recently an alternative method was given in "An Alternative to Null-Hypothesis Significance Tests" (Killeen 2005)
The statistic $P_{rep}$ estimates the probability of replicating an effect. It captures traditional publication criteria for signal-to-noise ratio, while avoiding parametric inference and the resulting Bayesian dilemma. In concert with effect size and replication intervals, $P_{rep}$ provides all of the information now used in evaluating research, while avoiding many of the pitfalls of traditional statistical inference.