# Determining whether a coin is fair

I have a dataset where an ostensibly 50% process has been tested 118 times and has come up positive 84 times.

My actual question:

• IF a process has a 50% chance of testing positive and
• IF you then run it 118 times
• What is the probability that you get AT LEAST 84 successes?

My gut feeling is that, the more tests are run, the closer to a 50% success rate I should get and so something might be wrong with the process (That is, it might not truly be 50%) but at the same time, it looks like it's running correctly, so I want to know what the chances are that it's actually correct and I've just had a long string of successes.

-
I'm guessing these were rolls online. ^_^ $\:$ – Ricky Demer Jun 30 '13 at 9:33
for further research you could look into p-value and null hypothesis testing. Basically, if you get your answer, like you did, as 71%, the p value will tell you "how likely" that the 71% is a fluke, and you are actually dealing with normal 50/50 coins the whole time. – Justin L. Jun 30 '13 at 9:41

The total number of successes in $n=118$ runs is binomial $(n,\frac12)$ hence the probability $p_n(k)$ to get at least $k=84$ successes is $$p_n(k)=2^{-n}\sum_{i=k}^n{n\choose i}.$$ When $k$ is significantly larger than $\frac{n}2$, $p_n(k)$ is very small and an estimation of how small $p_n(k)$ is is obtained through a large deviations estimate. This says that $p_n(k)\leqslant p_n^*(k)$ with $$p^*_n(k)=2^{-n}\inf\{(1+s)^ns^{-k}\,;\,s\geqslant1\}.$$ For every $k\gt\frac{n}2$, the infimum is reached at $s=\frac{k}{n-k}$, hence $$p^*_n(k)=2^{-n}n^nk^{-k}(n-k)^{-(n-k)}=\left(I\left(\tfrac{k}n\right)\right)^{-n},\quad I(t)=2t^t(1-t)^{1-t}.$$ For example, if $k=84$ and $n=118$, then $t=.712$ hence $I(t)\approx1.09710$ and $$p^*_{118}(84)\approx(1.09710)^{-118}\approx10^{-5}.$$ Numerically, $p_{118}(84)\approx2.36224\cdot10^{-6}$ and $p^*_{118}(84)\approx1.78153\cdot10^{-5}$.

-
At this point, I would like to add that I am not a statistician by trade (I'm sure I'd find this easier if I was) so I'm going to need som clarification - the parenthesis with one constant over another - that represents what, exactly? EDIT: Also, does the Large Deviations Estimate hold when k is only 42% larger than n/2 ? – medivh Jun 30 '13 at 9:23
${n\choose i}=\frac{n!}{i!(n-i)!}$ is a binomial coefficient. – Did Jun 30 '13 at 9:26
The upper bound in my post holds for every $n\geqslant1$ and every $k\geqslant n/2$. No $=$ or $\leqslant$ sign is an approximation, only the $\approx$ are (to numerical values). – Did Jun 30 '13 at 9:27
What really? Why is it called LARGE deviations then? I mean, in that case it would hold even if my process had run only 3 trials and come up 1, 1, 0. Or even if it had run 1001 trials and come up as 1 501 times and 0 500 times. (I realize you're not the one naming these but... "Large Deviations," really? – medivh Jun 30 '13 at 9:30
Yes, "large deviations" really. The method can always be applied but the upper bound it yields is not good when $k\gt n/2$, $k-n/2$ small (note that $I(1/2)=1$ and bounding a probability by $1$ is not a Herculean task...). On the other hand, if $n\to\infty$ and $k/n\to t$ with $t\gt1/2$, then $I(t)^{-n}$ is the correct order of exponential convergence to zero of $p_n(k)$ in the sense that $n^{-1}\log p_n(k)\to I(t)^{-1}$. The large deviations regime is when $(k/n)-1/2$ converges to a positive limit. If $k\gt n/2$, $k-n/2$ of order $\sqrt{n}$, one should rather use the central limit theorem. – Did Jun 30 '13 at 9:38

Of course, 118 is in the "small numbers regime", where one
can easily (use a computer to) calculate the probability exactly.

By wolframalapha,
the probability that you get at least 84 successes $\;\;=\;\; \frac{\displaystyle\sum_{s=84}^{118}\:\binom{118}s}{2^{118}}$

$=\;\; \frac{392493659183064677180203372911}{166153499473114484112975882535043072} \;\;\approx\;\; 0.00000236224 \;\;\;\;$.

-
Thank you, that should be enough to go from. – medivh Jun 30 '13 at 9:56
"Large numbers regime" does not refer to the possibility (or impossibility) to actually compute an exact value. – Did Jun 30 '13 at 9:57

Let $X\equiv$ number of times the process comes up positive in $n=118$ trials, where we observe that $x=84$. Then $X \sim \text{Binomial}(118,p)$, where $p$ represents the probability of a positive result. Our hypotheses are:

• $H_0: p=0.5$ (The process really is $50\%$.)
• $H_1: p \ne 0.5$ (The process actually isn't $50\%$.)

We now calculate our $p$-value to be: $$2Pr(X\ge84 \mid H_0 \text{ is true}) = 2\left[\sum_{k=84}^{118} \binom{118}{k}(0.5)^{118} \right] \approx 4.72447 \times 10^{-6}$$

Hence, since this $p$-value is much less than $\alpha=0.05$, we reject $H_0$ and conclude that there is strong evidence that the process actually isn't $50\%$.

-
The OP's "actual question" would call for a one-tailed test. $\:$ – Ricky Demer Jun 30 '13 at 9:49
@RickyDemer I was looking at the title ("Determining whether a coin is fair"), so I figured it would be two-tailed. Regardless, if we actually wanted it to be one-tailed, that would only make the $p$-value even smaller by a factor of $1/2$, giving us even further evidence to reject the null hypothesis. – Adriano Jun 30 '13 at 9:52