Area Under The Curve of CDF and Alpha Value - Chi-Sq Test Interpretation Can you help me understand the meaning of the area under the curve of a PDF in the interpretation of a Chi-Square test?
This is what I think I know:
If my test statistic (i.e. the chi-square value) increases for a given Degree of Freedom, so does my cumulative distribution function (CDF) value of the Chi-Square CDF(Chi). This value represents the area under the curve of the Chi-Square probability density function (PDF).
CDF(Chi) is also the significance level or "critical region" called $\alpha$ (often used with 0.05).
Is it correct, that if for a given test value the CDF(Chi) > 0.05 my H0 can be rejected when only interested in a one-tailed test - i.e. are two distributions different or not? This approach seems to work if I do a one-tailed test.
This is however currently counterintuitive to me, because as far as I understand, p = 1 - CDF(Chi) and H0 is rejected if p < 0.05. Therefore I would believe that the above statement is not true and I would need to reject H0 if CDF > 0.95. That would also make more sense to me, as it means that H0 is only rejected if there is more than 95% "evidence" that it is not random or less than a 5% chance of wrongfully rejecting H0.
Can someone help clear things up - I assume my last thought makes more sense and in my example, I am getting results that seem to conform to what I want to see but are actually not statically significant.
Thanks!
 A: Consider the following fictitious counts for four
categories, where the first category is half as
likely as the other three. Thus, we would suspect
that, with enough observations, a chi-squared
test of the null hypothesis--that categories are
equally likely--will be rejected.
set.seed(205)
x = sample(1:4, 100, rep=T, p=c(1,2,2,2))
table(x)
x
 1  2  3  4 
14 34 25 27 

Notice that the first category does indeed have
a lower count out of 100 observations than do
the other three categories.
With 100 observations under the null hypothesis,
the expected count for each category is $25.$
You can check for yourself that the chi-squared
statistic is $8.24.$ (One of the observed counts is 'too
small' and three are 'too large', but differences are squared, so all four terms in the statistic are positive:
$\sum_{i=1}^4 \frac{(X_i-25)^2}{25} = 8.24.)$
Because the chi-squared statistic is approximately
distributed as $\mathsf{Chisq}(\nu = 4-1=3)$ when $H_0$ is true, the
critical value of the test is $c = 7.815,$ which
cuts probability 5% from the upper tail of
$\mathsf{Chisq}(\nu = 3),$
Because the chi=squared
statistic is $8.24 > 7.815,$ the agreement
between observed and expected counts is worse
than would be expected by chance, and the null
hypothesis is rejected at the 5% level of significance.
(Remember that the chi-squared statistic really
measures badness of fit: the larger the statistic,
the worse the fit of the data to the null hypothesis.)
 c = qchisq(.95, 3);  c
 [1] 7.814728

The P-value of this chi-squared test is $0.041,$
the probability in the right tail of
$\mathsf{Chisq}(\nu=3)$ beyond the observed
value $8.24.$  We reject the null hypothesis
at the 5% level if the P-value is smaller than $0.05.$
1 - pchisq(8.24, 3)
[1] 0.04130349

In R, the formal chi-squared test looks like this. (When ho hypothetical probabilities are specified,
the default null hypothesis is that all categories are equally likely.)
chisq.test(table(x))

        Chi-squared test 
        for given probabilities

data:  table(x)
X-squared = 8.24, df = 3, p-value = 0.0413

The figure below shows the density function of
$\mathsf{Chisq}(\nu 3).$ The vertical blue line
is at the observed value of the chi-square statistic;
the area to the right of it under the curve is the
P-value. The vertical dotted red line is at the
critical value $c;$ the area to the right of it under the density curve is 5%.

