# Relationship between chi-square goodness-of-fit testing and chi-square distribution?

Anyone care to explain the relationship between a Chi-square goodness of fit test and the Chi-square distribution? One has the expected value in the denominator and the other has the variance in the denominator, but everything I find just kinda glosses over that.

Most explanations for goodness of fit use the expectation value in the denominator (such as in the link below) and then use the variance when talking about the chi-square distribution independent of goodness of fit testing. http://www.stat.yale.edu/Courses/1997-98/101/chigf.htm

But then this one uses the variance in the denominator for goodness of fit testing.... http://maxwell.ucsc.edu/~drip/133/ch4.pdf

Suppose our model is a distribution based on categories $A_1, A_2,$ and $A_3$, with probabilities $P(A_1) = \theta_1 = 1/4,$ $P(A_2) = \theta_2 = 1/4,$ and $P(A_3) = \theta_3 = 1/2.$ This multinomial model is our null hypothesis.

Also suppose we have $n$ observations with observed counts $X_1, X_2, X_3$ in the respective categories. As an approximation, we view $X_i$ as $Poisson(\theta_i),$ for $i = 1,2,3=K,$ respectively. The Poisson means are $\lambda_i = n\theta_i$ and their variances are also $\lambda_i = n\theta_i.$ Then the three standardized Poisson random variables are $Z_i = \frac{X_i - \lambda_i}{\sqrt{\lambda_i}}.$ For sufficiently large $n,$ the $Z_i$ are approximately standard normal.

If the $Z_i$ were (i) truly normal and (ii) independent, we would have

$$Q = \sum_{i=1}^K Z_i^2 = \sum_i \frac{(X_i - \lambda_i)^2}{\lambda_i} \sim Chisq(df=3).$$

This random variable $Q$ is the chi-squared goodness-of-fit statistic. The $\lambda_i$ are often called the 'expected' counts and the $X_i$ the 'observed' counts. Relevant to your Question is that the $\lambda_i$ in the denominator are, by coincidence, also the Poisson variances. Thus what may be an apparent conflict in notation is not real.

Lack of independence is due to the fact that $\sum_i X_1 = n.$ Through an argument I will skip here, this lack of independence is taken into account by reducing the degrees of freedom from $K = 3$ to $K - 1 = 2.$ Various rules are supposed to take care of the approximation of Poisson to normal with increasing $n$. (For example, some authors say that all $K$ of the $\lambda_i$ should exceed 3, more fussy authors say 5.)

In practice the statistic $Q$ taken to be $Chisq(K-1)$ works quite well. One might say especially well because, in testing our null hypothesis, we are not interested in the fit of the whole distribution of $Q$ to $Chisq(K-1)$, but mainly in the fit in the tail beyond about the 90th percentile. Bear in mind, however, than $Q$ inherits discreteness from the $X_i$ and the chi-squared distribution is continuous, so exact agreement is not possible.

The simulation below investigates the fit of the actual $Q$ to the approximating chi-squared distribution, using a small enough $n$ that the fit might reasonably be in some doubt.

 m = 10^5;  q = numeric(m);  th = c(1/4, 1/4, 1/2);  n = 25
for (i in 1:m) {
s = sample(1:3, n, repl=T, prob = th)
x = c(sum(s==1), sum(s==2), sum(s==3))
q[i] = sum((x-n*th)^2/(n*th)) }
mean(q > qchisq(.95, 3-1))
##   0.04818


So a nominal test at level 5% using $Chisq(2)$ has actual significance level 4.8%. (Among the 100,000 simulated performances of the experiment, fewer than 100 distinct values of $Q$ were encountered. The histogram below omits results from about 0.02% of the iterations producing values $Q$ in the far right tail.)

This question is answered here, which is itself a reference to here

The proof there illustrates how the constraint among the K $Z^2$ variables(sum of observations = N), implies they are not independent, but can can be transformed into K-1 independent standard Normal random variables (all in the limit). In other words, using the mean rather than the variance in the denominators comes along with reducing the number of degrees of freedom.

Comment on above answer: The answer above says the bin count can be approximated as Poisson, so the mean equals the var. But that approximation is for large n, small p. It's actually binomial where the mean is not equal var: mean = np, var = np(1-p), where, again for small p, var = np = mean.