2
$\begingroup$

Can you help me understand the meaning of the area under the curve of a PDF in the interpretation of a Chi-Square test?

This is what I think I know:

If my test statistic (i.e. the chi-square value) increases for a given Degree of Freedom, so does my cumulative distribution function (CDF) value of the Chi-Square CDF(Chi). This value represents the area under the curve of the Chi-Square probability density function (PDF).

CDF(Chi) is also the significance level or "critical region" called $\alpha$ (often used with 0.05).

Is it correct, that if for a given test value the CDF(Chi) > 0.05 my H0 can be rejected when only interested in a one-tailed test - i.e. are two distributions different or not? This approach seems to work if I do a one-tailed test.

This is however currently counterintuitive to me, because as far as I understand, p = 1 - CDF(Chi) and H0 is rejected if p < 0.05. Therefore I would believe that the above statement is not true and I would need to reject H0 if CDF > 0.95. That would also make more sense to me, as it means that H0 is only rejected if there is more than 95% "evidence" that it is not random or less than a 5% chance of wrongfully rejecting H0.

Can someone help clear things up - I assume my last thought makes more sense and in my example, I am getting results that seem to conform to what I want to see but are actually not statically significant.

Thanks!

$\endgroup$
1
  • $\begingroup$ A traditional chi-squared test is fundamentally one sided, if the chi-squared statistic (based on the data) is large, that indicates that the observed frequencies are a poor fit to the expected frequencies. // Maybe some of the 'Related' links in the margin will be helpful, but I am including my own Answer below, which I hope answers your questions. $\endgroup$
    – BruceET
    Feb 6, 2022 at 0:15

1 Answer 1

1
$\begingroup$

Consider the following fictitious counts for four categories, where the first category is half as likely as the other three. Thus, we would suspect that, with enough observations, a chi-squared test of the null hypothesis--that categories are equally likely--will be rejected.

set.seed(205)
x = sample(1:4, 100, rep=T, p=c(1,2,2,2))
table(x)
x
 1  2  3  4 
14 34 25 27 

Notice that the first category does indeed have a lower count out of 100 observations than do the other three categories.

With 100 observations under the null hypothesis, the expected count for each category is $25.$ You can check for yourself that the chi-squared statistic is $8.24.$ (One of the observed counts is 'too small' and three are 'too large', but differences are squared, so all four terms in the statistic are positive: $\sum_{i=1}^4 \frac{(X_i-25)^2}{25} = 8.24.)$

Because the chi-squared statistic is approximately distributed as $\mathsf{Chisq}(\nu = 4-1=3)$ when $H_0$ is true, the critical value of the test is $c = 7.815,$ which cuts probability 5% from the upper tail of $\mathsf{Chisq}(\nu = 3),$

Because the chi=squared statistic is $8.24 > 7.815,$ the agreement between observed and expected counts is worse than would be expected by chance, and the null hypothesis is rejected at the 5% level of significance. (Remember that the chi-squared statistic really measures badness of fit: the larger the statistic, the worse the fit of the data to the null hypothesis.)

 c = qchisq(.95, 3);  c
 [1] 7.814728

The P-value of this chi-squared test is $0.041,$ the probability in the right tail of $\mathsf{Chisq}(\nu=3)$ beyond the observed value $8.24.$ We reject the null hypothesis at the 5% level if the P-value is smaller than $0.05.$

1 - pchisq(8.24, 3)
[1] 0.04130349

In R, the formal chi-squared test looks like this. (When ho hypothetical probabilities are specified, the default null hypothesis is that all categories are equally likely.)

chisq.test(table(x))

        Chi-squared test 
        for given probabilities

data:  table(x)
X-squared = 8.24, df = 3, p-value = 0.0413

The figure below shows the density function of $\mathsf{Chisq}(\nu 3).$ The vertical blue line is at the observed value of the chi-square statistic; the area to the right of it under the curve is the P-value. The vertical dotted red line is at the critical value $c;$ the area to the right of it under the density curve is 5%.

enter image description here

$\endgroup$
3
  • 1
    $\begingroup$ Thanks, Bruce for the detailed answer. To make sure I got it - when chi-sq value is smaller than 7.8147 for df=3 I will need to reject my H0. Also, if my cumulative distribution function returns a value greater than 0.95 I would reject H0 as that means that my chi-square value is on the right side of my critical value aka. the area under the curve on the right to my critical value is smaller than 0.05 aka there is less than 5% probability of my values being equally likely. $\endgroup$
    – copcor
    Feb 6, 2022 at 18:44
  • 1
    $\begingroup$ (a). Reject when chi-squared statistic $8.24$ exceeds critical value $c = 7.8147$ (red dotted line). The static is at the heavy blue line and so you reject. (b) Yes, the CDF of CHISQ(3) at $8.25$ is $0.9586965 > 0.95$ as it should, leaving less than 5% beyond $8.25$ in the right tail. // Suggest you master use of critical values (shown in printed tables) first. Then tackle P-values (shown in computer printout). $\endgroup$
    – BruceET
    Feb 6, 2022 at 20:43
  • 1
    $\begingroup$ Apologies - I meant larger than the critical value. Thanks! $\endgroup$
    – copcor
    Feb 7, 2022 at 2:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .