Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I have a dataset where an ostensibly 50% process has been tested 118 times and has come up positive 84 times.

My actual question:

  • IF a process has a 50% chance of testing positive and
  • IF you then run it 118 times
  • What is the probability that you get AT LEAST 84 successes?

My gut feeling is that, the more tests are run, the closer to a 50% success rate I should get and so something might be wrong with the process (That is, it might not truly be 50%) but at the same time, it looks like it's running correctly, so I want to know what the chances are that it's actually correct and I've just had a long string of successes.

share|improve this question
    
I'm guessing these were rolls online. ^_^ $\:$ –  Ricky Demer Jun 30 '13 at 9:33
2  
for further research you could look into p-value and null hypothesis testing. Basically, if you get your answer, like you did, as 71%, the p value will tell you "how likely" that the 71% is a fluke, and you are actually dealing with normal 50/50 coins the whole time. –  Justin L. Jun 30 '13 at 9:41

3 Answers 3

up vote 2 down vote accepted

The total number of successes in $n=118$ runs is binomial $(n,\frac12)$ hence the probability $p_n(k)$ to get at least $k=84$ successes is $$ p_n(k)=2^{-n}\sum_{i=k}^n{n\choose i}. $$ When $k$ is significantly larger than $\frac{n}2$, $p_n(k)$ is very small and an estimation of how small $p_n(k)$ is is obtained through a large deviations estimate. This says that $p_n(k)\leqslant p_n^*(k)$ with $$ p^*_n(k)=2^{-n}\inf\{(1+s)^ns^{-k}\,;\,s\geqslant1\}. $$ For every $k\gt\frac{n}2$, the infimum is reached at $s=\frac{k}{n-k}$, hence $$ p^*_n(k)=2^{-n}n^nk^{-k}(n-k)^{-(n-k)}=\left(I\left(\tfrac{k}n\right)\right)^{-n},\quad I(t)=2t^t(1-t)^{1-t}. $$ For example, if $k=84$ and $n=118$, then $t=.712$ hence $I(t)\approx1.09710$ and $$ p^*_{118}(84)\approx(1.09710)^{-118}\approx10^{-5}. $$ Numerically, $p_{118}(84)\approx2.36224\cdot10^{-6}$ and $p^*_{118}(84)\approx1.78153\cdot10^{-5}$.

share|improve this answer
    
At this point, I would like to add that I am not a statistician by trade (I'm sure I'd find this easier if I was) so I'm going to need som clarification - the parenthesis with one constant over another - that represents what, exactly? EDIT: Also, does the Large Deviations Estimate hold when k is only 42% larger than n/2 ? –  medivh Jun 30 '13 at 9:23
    
${n\choose i}=\frac{n!}{i!(n-i)!}$ is a binomial coefficient. –  Did Jun 30 '13 at 9:26
    
The upper bound in my post holds for every $n\geqslant1$ and every $k\geqslant n/2$. No $=$ or $\leqslant$ sign is an approximation, only the $\approx$ are (to numerical values). –  Did Jun 30 '13 at 9:27
    
What really? Why is it called LARGE deviations then? I mean, in that case it would hold even if my process had run only 3 trials and come up 1, 1, 0. Or even if it had run 1001 trials and come up as 1 501 times and 0 500 times. (I realize you're not the one naming these but... "Large Deviations," really? –  medivh Jun 30 '13 at 9:30
    
Yes, "large deviations" really. The method can always be applied but the upper bound it yields is not good when $k\gt n/2$, $k-n/2$ small (note that $I(1/2)=1$ and bounding a probability by $1$ is not a Herculean task...). On the other hand, if $n\to\infty$ and $k/n\to t$ with $t\gt1/2$, then $I(t)^{-n}$ is the correct order of exponential convergence to zero of $p_n(k)$ in the sense that $n^{-1}\log p_n(k)\to I(t)^{-1}$. The large deviations regime is when $(k/n)-1/2$ converges to a positive limit. If $k\gt n/2$, $k-n/2$ of order $\sqrt{n}$, one should rather use the central limit theorem. –  Did Jun 30 '13 at 9:38

Of course, 118 is in the "small numbers regime", where one
can easily (use a computer to) calculate the probability exactly.



By wolframalapha,
the probability that you get at least 84 successes $\;\;=\;\; \frac{\displaystyle\sum_{s=84}^{118}\:\binom{118}s}{2^{118}}$

$=\;\; \frac{392493659183064677180203372911}{166153499473114484112975882535043072} \;\;\approx\;\; 0.00000236224 \;\;\;\; $.

share|improve this answer
    
Thank you, that should be enough to go from. –  medivh Jun 30 '13 at 9:56
    
"Large numbers regime" does not refer to the possibility (or impossibility) to actually compute an exact value. –  Did Jun 30 '13 at 9:57

Let $X\equiv$ number of times the process comes up positive in $n=118$ trials, where we observe that $x=84$. Then $X \sim \text{Binomial}(118,p)$, where $p$ represents the probability of a positive result. Our hypotheses are:

  • $H_0: p=0.5$ (The process really is $50\%$.)
  • $H_1: p \ne 0.5$ (The process actually isn't $50\%$.)

We now calculate our $p$-value to be: $$ 2Pr(X\ge84 \mid H_0 \text{ is true}) = 2\left[\sum_{k=84}^{118} \binom{118}{k}(0.5)^{118} \right] \approx 4.72447 \times 10^{-6} $$

Hence, since this $p$-value is much less than $\alpha=0.05$, we reject $H_0$ and conclude that there is strong evidence that the process actually isn't $50\%$.

share|improve this answer
1  
The OP's "actual question" would call for a one-tailed test. $\:$ –  Ricky Demer Jun 30 '13 at 9:49
    
@RickyDemer I was looking at the title ("Determining whether a coin is fair"), so I figured it would be two-tailed. Regardless, if we actually wanted it to be one-tailed, that would only make the $p$-value even smaller by a factor of $1/2$, giving us even further evidence to reject the null hypothesis. –  Adriano Jun 30 '13 at 9:52

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.