# $\chi^2$ tests giving apparently conflicting results; and debate on validity of classical statistics

I've come across some very bright people who say that "classical statistics" is "plain wrong", as is also said about cosmology based on string theory. I think they prefer the "Bayesian" statistics that classical statistics replaced, or at least, most of them do. I'm looking for a way in to understanding what they are talking about! Can people please help with the following question, on which I think reflection may be useful in this connection.

A random sample of 59 people yielded the following results:

                    Eye-colour
1(Blue)   2(Brown)
------------------
Sex 1 (Male)       19       10
2 (Female)      9       21


Carry out Pearson’s $\chi^2$-squared test at the 5% level of the null hypothesis that sex and eye-colour are independent. Now do the same for the null hypothesis that each of the cell probabilities is $\frac{1}{4}$.

In the first case, I get $T=7.46 > \chi_1^2(0.05) = 3.84$. So reject the NH.
In the second case, I get $T=7.64 < \chi_3^2(0.05) = 7.81$. So accept the NH.

We appear to be getting an absurdity; that we should reject the hypothesis that the variables are independent, but accept the hypothesis that each of the probabilities is $\frac{1}{4}$, which implies independence.

Assistance would be welcome!

• Why would the Bayesian interpretation make a difference here? In fact, what is the Bayesian interpretation, since all the computations done are founded on the frequentist approach? Also, note that if you choose a more stringent $\alpha$, you won't have the problem. – Raskolnikov Jun 17 '13 at 9:28

On the philosophical issue: The controversy between "frequentists" and "bayesians" is perhaps one of the most misguided scientific controversies of all time (although it was useful because it made both sides work hard on their cases, thus refining results an producing many new ones). This is not a site for opinions, but since you brought it up, I will record mine: the problem started because for centuries, scholars treated "probability" as an entity of the real world. So they thought that they were trying to understand and interpret a real-world phenomenon -and when you do that, for logical consistency, necessarily only one interpretation must be right. Hence the fight: frequentists said that probability is the limit of empirical frequency, bayeasians said that probability is a subjective belief. Great mind Kolmogorov finally solved the issue in the 1930's, but still, it took half a century to really sink in: Kolmogorov said: "probability is an abstract mathematical concept, a set function, a measure, which I axiomatize in this and that way. Now take this abstract mathematical concept and see what phenomena of the real world can be usefully modeled and studied by using it". And lo: empirical frequencies are real-world phenomena, they are observed and can be measured. But subjective beliefs are also real-world phenomena: people have them, it is a fact. And the mathematical concept of probability can be usefully employed to model and study both. And there is no problem, no issue, no controversy over that -after all, this is what we do with mathematical concepts - we apply the same concept (think "function") to model and study very distinct real-world phenomena, because they share some common structure. Narrow-mindedness is uniformly distributed independent of anything else... which brings us to your question.  About the question: First, let's make sure we understand the same things: The table you present contains (absolute) empirical frequencies. If you divide them by the sample size, you get the relative empirical frequencies, which are linked to the concept of probabilities, let's call them that from now on. Let's say you create the table with the empirical probabilities... but what probabilities are they? They are joint probabilities: the probabilities that two "events" (here eye-color and sex) "happen together". For example in the upper left corner you would have $$P(male \land blue)= \frac {19} {59}$$
What is the definition of independence? That the joint probability equals the product of the marginal (stand-alone) probabilities. For the upper left corner this would require $P(male \land blue) = P(male)P(blue)$ and analogously for all for entries. Now, this does not imply that independence obtains only if all theoretical joint probabilities are equal. To see this define the following (I don't know how to do this in table format but I follow the structure of your table) $$\text{Absolute Frequency} (male \land blue)= a$$ $$\text{Absolute Frequency} (male \land brown)= b$$ $$\text{Absolute Frequency} (female \land blue)= c$$ $$\text{Absolute Frequency} (female \land brown)= d$$
This are joint absolute frequencies. Our sample size is $a+b+c+d$. If you construct this as a table and then divide by the sample size you will find that the (marginal) probabilities are $$P(male)= \frac{a+b}{a+b+c+d}$$ $$P(female)= \frac{c+d}{a+b+c+d}$$ $$P(blue)= \frac{a+c}{a+b+c+d}$$ $$P(brown)= \frac{b+d}{a+b+c+d}$$
Now we want to see under which condition(s) independence will hold. For example, we want the joint probability of female sex and brown eyes to equal the product of the (marginal) probability of female times the (marginal) probability of brown eyes. Namely we must have $$P(female \land brown)= P(female)P(brown) \;$$ $$\Rightarrow \frac {d} {a+b+c+d}= \frac {c+d} {a+b+c+d}\frac {b+d} {a+b+c+d}$$ If you carry out the algebra you will find the condition: $$\text{Independence if and only if}\; ad=bc$$ ...in a probabilistic, hypothesis testing framework. And you will get the same result for all four entries. It is obvious, that $ad=bc$ does not require necessarily $a=b=c=d$ (although this is one possibility). I guess you can now look again at how you implemented the chi-square tests.
If sex and eye colour are independent, the chance of having lots of males with blue eyes and a similar number of females with brown eyes is highest when the probabilities are all $\frac{1}{4}$. If the probabilities are close enough to $\frac{1}{4}$ for the data to not appear significantly different at the $5\%$ level, then the probability of getting the results above is highest when sex and eye colour are dependent.