2
$\begingroup$

I'm studying the chi-square test of independence.

According to my understanding, we first hypothesize independence between variables and consider them as being normally distributed. Then we go on to calculate the test statistic as

$$ Z_i = \frac{O_i-E_i}{\sqrt{E_i}}$$ where O is the observed value from table and E is the calculated expectation.

  1. Is $Z_i \sim \mathcal{N}(0,1)$ ?
  2. Is $\sum_1^k Z_i^2 \sim \chi^2(k)$ ?
$\endgroup$
  • $\begingroup$ I believe the answer to 1 is no $X=\frac{O-E}{\sqrt{E}}$ is approximately normal. If $Z\sim N(0,1)$ then the definition you have for $\chi^2$ is correct but note that this is not the same as $\sum \limits_1^k X_i^2$ I lack the technical knowledge to explain further but I'd be interested in a clear explanation myself $\endgroup$ – Karl Apr 28 '15 at 10:24
  • $\begingroup$ Thanks karl. I have changed $Z$ to $Z_i$ to reflect different random variables. $\endgroup$ – ZeroCool Apr 28 '15 at 11:19
  • $\begingroup$ I asked a question on cross validated that is still open and I believe relevant. I would link it here but I don't know how. I have found it very difficult to find an explanation I can understand. They are usually here's the test just do it (I have no interest in) or far too advanced. I began to look at Cochrans theorem and understood a little but I'd ideally like something to prerequisite this. $\endgroup$ – Karl Apr 28 '15 at 12:42
2
$\begingroup$

Continuous distributions. First, if $Z \sim N(0,1)$, then $Z^2 \sim Chisq(df=1)$. This is easily proved using direct integration or moment generating functions. Next, if $Z_1, \dots Z_k$ are independently $N(0,1),$ then $\sum_{i=1}^k X_i^2 \sim Chisq(df=k),$ as you say. This is easily proved by noticing the $k$th power of the MGF of $Chisq(1)$ is the MGF of $Chisq(k)$.

Discrete distributions.However, the above results to not apply directly to your question about a chi-squared test of independence or of goodness-of-fit. In this case the $O_i$ are integers and so your $Z_i$ are necessarily discrete (although not generally integer valued). So the distribution of $Z_i$ is at best only approximately $N(0,1),$ so $Z_i^2$ is at best approximately $Chisq(1).$

The usual assumption thus far is that the $O_i$ are Poisson with means $E_i$ and standard deviation $\sqrt{E_i}.$ Accordingly, your $Z_i$ are standardized Poissons, which are approximately standard normal for sufficiently large $E_i.$

Goodness-of-Fit tests. A very simple goodness-of-fit test would be to roll a fair die $n=600$ times so that there are $n$ possible values $1, \dots, 6,$ each with probability $p_i = 1/6$ and $E_i = np_i = 600(1/6) = 100.$ Here, the $O_i$ are the actual number of times each value (face) appears. They are actually binomial, approximately Poisson, and approximately normal.

However, in this case, $\sum_{i=1}^k Z_i^2 \sim Chisq(n-1).$ There are $n-1$ degrees of freedom instead of $n$, roughly speaking, because there is one linear constraint: the $O_i$ must sum to 600.

Tests if Independence. In a typical chi-squared test of independence there are two categorical variables and we are trying to judge their independence. This is not the place for a full review of such tests, but we can briefly state the approximate distribution theory.

If the 'contingency table' is based on $r$ rows and $c$ columns (corresponding to levels of the two categorical variables), then you have the test statistic $Q = \sum_{ij} (O_{ij} - E_{ij})^2 /E_{ij},$ where the $E_{ij}$ are computed assuming independence of the two categorical variables. Then $Q$ is approximately distributed as $Chisq((r-1)(c-1)),$ provided the $E_{ij}$ are sufficiently large. Again here, the number of degrees of freedom is adjusted downward from $cr$ to account for the linear constraints imposed by the structure of the computations.

Simulation studies have shown that the fit to a chi-squared distribution is "sufficiently good" in the right-hand tail (where accept/reject decisions are made), if all the $E_{ij} > 3$ and "all but a few" have $E_{ij} > 5.$

$\endgroup$
  • 1
    $\begingroup$ Thanks for the effort. I had thought about the binomial approximately poison approximately normal but could never find a decent reference. Could you suggest any please? $\endgroup$ – Karl Apr 28 '15 at 21:29
  • $\begingroup$ @Karl: Google 'chi-squared goodness-of-fit'. Look at Wikipedia and Yale for some of the facts. Discussions of reasons for DF are scarce because they are based on multi-dimensional limit theorems for conditional distributions. Information about degree of approx. of statistic to ch-sq dist'n is mainly from simulation studies. I could append a brief simulation for the dice example if you think it would be a useful addition. $\endgroup$ – BruceET Apr 28 '15 at 21:56
  • $\begingroup$ Thank you very much Bruce, this explains all the theory! Very appreciated. $\endgroup$ – ZeroCool Apr 30 '15 at 21:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.