Determining whether a coin is fair I have a dataset where an ostensibly 50% process has been tested 118 times and has come up positive 84 times.
My actual question:


*

*IF a process has a 50% chance of testing positive and

*IF you then run it 118 times

*What is the probability that you get AT LEAST 84 successes?


My gut feeling is that, the more tests are run, the closer to a 50% success rate I should get and so something might be wrong with the process (That is, it might not truly be 50%) but at the same time, it looks like it's running correctly, so I want to know what the chances are that it's actually correct and I've just had a long string of successes.
 A: Of course, 118 is in the "small numbers regime", where one

can easily (use a computer to) calculate the probability exactly.

By wolframalapha,

the probability that you get at least 84 successes $\;\;=\;\; \frac{\displaystyle\sum_{s=84}^{118}\:\binom{118}s}{2^{118}}$
$=\;\; \frac{392493659183064677180203372911}{166153499473114484112975882535043072} \;\;\approx\;\; 0.00000236224 \;\;\;\; $.
A: The total number of successes in $n=118$ runs is binomial $(n,\frac12)$ hence the probability $p_n(k)$ to get at least $k=84$ successes is
$$
p_n(k)=2^{-n}\sum_{i=k}^n{n\choose i}.
$$
When $k$ is significantly larger than $\frac{n}2$, $p_n(k)$ is very small and an estimation of how small $p_n(k)$ is is obtained through a large deviations estimate. This says that $p_n(k)\leqslant p_n^*(k)$ with
$$
p^*_n(k)=2^{-n}\inf\{(1+s)^ns^{-k}\,;\,s\geqslant1\}.
$$
For every $k\gt\frac{n}2$, the infimum is reached at $s=\frac{k}{n-k}$, hence
$$
p^*_n(k)=2^{-n}n^nk^{-k}(n-k)^{-(n-k)}=\left(I\left(\tfrac{k}n\right)\right)^{-n},\quad I(t)=2t^t(1-t)^{1-t}.
$$
For example, if $k=84$ and $n=118$, then $t=.712$ hence $I(t)\approx1.09710$ and 
$$
p^*_{118}(84)\approx(1.09710)^{-118}\approx10^{-5}.
$$
Numerically, $p_{118}(84)\approx2.36224\cdot10^{-6}$ and $p^*_{118}(84)\approx1.78153\cdot10^{-5}$.
A: Let $X\equiv$ number of times the process comes up positive in $n=118$ trials, where we observe that $x=84$. Then $X \sim \text{Binomial}(118,p)$, where $p$ represents the probability of a positive result. Our hypotheses are:


*

*$H_0: p=0.5$ (The process really is $50\%$.)

*$H_1: p \ne 0.5$ (The process actually isn't $50\%$.)


We now calculate our $p$-value to be:
$$
2Pr(X\ge84 \mid H_0 \text{ is true}) = 2\left[\sum_{k=84}^{118} \binom{118}{k}(0.5)^{118} \right] \approx 4.72447 \times 10^{-6}
$$
Hence, since this $p$-value is much less than $\alpha=0.05$, we reject $H_0$ and conclude that there is strong evidence that the process actually isn't $50\%$.
