# How do we arrive at the conclusion that P(Head) =0.5 for a fair coin?

In Feynman's 'Lectures on Physics', I read a chapter on probability which tells that P(Head) for a fair coin 'approaches' 0.5 as no. of trials that we take goes to infinity (well, I tossed the coin 50 times & got heads 17 times, instead of 25 :-) ...). Can someone elaborate?

-
The definition of a "fair coin" is that it is equally likely to fall heads and tails (and a miniscule likelihood of landing on its edge and staying there). That means, the assumption is that $P(Head) = 0.5$. Experimentally, the probability of landing heads is the number of successful outcomes divided by the number of experiments; so if you perform $n$ trials, and compute $h/n$ ($h$ the number of heads), you expect $h/n\to P(h)$ as $n\to\infty$. $n=25$ is very far from $\infty$, of course... – Arturo Magidin Apr 5 '11 at 15:44
so the general assumption that P(H)=P(T)=0.5 is taken just for the sake of brevity or what? – Amit L Apr 5 '11 at 15:50
@Amit: Again: by definition, a "fair coin" is one in which $P(H)=P(T)$. Assuming that $P(E)$ is negligible (landing on its edge), which is reasonable for practical purposes, this gives $P(H)=P(E)=0.5$. But probability of 1/2 does not mean that in any particular experiment you will always get half the coin tosses heads and half tails; it means that in the long run you expect to get as many heads as tails. That is, if a coin is "fair" (under the above definition), and you perform an experiment with $n$ tosses, you expect $h/n$ to be "close to 0.5", with "how close" proportional to $1/n$. – Arturo Magidin Apr 5 '11 at 15:53
If you want to derive this from physical laws, as input you'll need two main ingredients: that the coin is symmetrical (of course this would give you the problem that you couldn't determine heads from tails, but ignore this!) and that there's no probability the coin could land in any configuration other than heads or tails -- say the "edge" of the coin is tapered to make standing on edge an unstable configuration. Then you compute the probability of landing in either configuration as the relative volume of the attractive basins (in state-space) for the two final configurations. – Ryan Budney Apr 5 '11 at 16:00
Get it more clearly now. After all we're talking about 'Probability' (& NOT 'Surety'). Hence, not getting 25 heads in my experiment of 50 tosses was not at all wrong result (to be lost in) or something. Thank you again, sir. And, +1 for "how close is proportional to 1/n" – Amit L Apr 5 '11 at 16:02

It is implied by the law of large numbers - the average sum of i.i.d. random variables (e.g. tosses of the fair coin) goes to the expectation.

-
what do you mean by i.i.d random variables? – Amit L Apr 5 '11 at 15:57
@Amit L: To expand on Gortaur's answer (adapted to our context), let $X_1,X_2,\ldots$ be a sequence of independent and identically distributed (i.i.d) random variables, such that ${\rm P}(X_i=0)={\rm P}(X_i=1)=1/2$. The expectation of $X_i$ is thus $\mu:={\rm E}(X_i)=1/2$. By the strong law of large numbers, the average $\bar X_n : = \frac{1}{n}\sum\nolimits_{i = 1}^n {X_i }$ converges with probability $1$ to $\mu = 1/2$. – Shai Covo Apr 5 '11 at 16:00
so is i.i.d by any means similar to uniform distribution of random variable? – Amit L Apr 5 '11 at 16:07
@Amit L: You can have i.i.d. random variables from any distribution: normal, exponential, geometric, uniform,... – Shai Covo Apr 5 '11 at 16:10
Indepedent means that if that the values of a variable are not influenced in any way by the outcomes of all the other variables. A classic example is that a biker outside your house does not influence your coin tosses. Identically distributed of course means that all the variables follow the same distribution, with the same parameters (mean,variance etc.) – chazisop Apr 5 '11 at 16:19

Everyone's answering this mathematically. I think a better answer is experimental. Andrew Gelman has referred to biased coins as the unicorn of probability theory; see also this paper by Andrew Gelman and Deborah Nolan. The basic idea is that coin tossing is a deterministic process, and the randomness comes from our uncertainty in the initial conditions; half the possible initial conditions lead to heads and half to tails. To bias a coin to come up heads, it would have to slow down in midair when heads was facing up and speed up when tails is facing up. Unless you have installed some sort of rocket boosters on your coin this is not possible.

-
can you please comment on 'coin toss is a deterministic process'? Thank you. – Amit L Apr 5 '11 at 16:20
@Michael: "Unless you have installed some sort of rocket boosters on your coin this is not possible." For real world examples, I believe if you make a coin such that one side is heavier than the other, it will be biased and have a higher probability that the lighter side appears facing up. – Eric Naslund Apr 5 '11 at 16:25
Amit: whether a coin comes up heads depends only on the position and orientation that it has when it leaves your hand and the velocity and angular momentum that you give it. (I may not have the list exactly right, but the point is it's some short list of classical physical quantities.) If we knew these quantities exactly we could tell in advance whether the coin will land heads or tails. – Michael Lugo Apr 5 '11 at 16:54
@Michael Lugo: Actually, according to work of Persi Diaconis and others, it's hard to remove the bias from the initial orientation of the coin. If you start the coin with the head up, and rotate about an axis perpendicular to the cylinder's axis, then this should remove the bias. However, if you are off by a few degrees, then the coin will not have heads up only half of the time. As an extreme example, imagine that you toss the coin up but spin it about the cylinder's axis. Most tosses are between the extremes, so they are biased. – Douglas Zare Apr 5 '11 at 19:42
Why am I not surprised that Diaconis has worked on this? – Michael Lugo Apr 5 '11 at 21:52

It is entirely plausible that your coin is not fair. But then again, going back to your little experiment, the probability that a FAIR coin tossed 50 times has 17 heads and 33 tails is FINITE. Which means it can occur.

-
I can continue experimenting on daily basis noting downs heads & tails to eventually tell my grandsons that the P(Heads) has surely approached 0.5 (from whichever side, I mean LHL or RHL)...:-) – Amit L Apr 5 '11 at 16:31
@Amit: Im not sure what you are saying, but you can calculate what P(H) is with the data you have, and you can say how accurate that value is. For example, right now, with your experiment, you have that $P(H) \approx 17/50$, then you can also say what the error in that value is (more complicated, so I am not going to explain it en.wikipedia.org/wiki/Error_analysis) – picakhu Apr 5 '11 at 16:34
@Amit: Alternatively, you can "toss" it more times. The probability that you are far from the the actual 0.5 decreases with the number of tosses, let me know if you want to see a proof of the law of large numbers. – picakhu Apr 5 '11 at 16:37
I actually gotta see this 'Law Of Large Numbers'. Just a one-liner (say-it-all kinda), please... – Amit L Apr 5 '11 at 16:42
@Amit: I do not know how to explain it in one line. Let me give you the Wiki reference. en.wikipedia.org/wiki/Law_of_large_numbers The graph there is actually really nice. Notice how as the number of trials gets bigger, the average reaches the expected average. – picakhu Apr 5 '11 at 16:43

First of all, keep in your mind that probability is a tool of mathematics. Although you can apply mathematics in the real world, that does not mean that everything true in mathematics is true in the real world as well. This works in the opposite direction as well.

A fair coin is a mathematical abstraction that is defined as a coin that when tossed has a probability of $0.5$ of landing on heads and an equal probability of landing on tails, thus the name "fair". You define it that way and it is automatically true. Building a truly fair coin in the real world would require a ridiculous amount of time and perhaps nanotechnology that we do not have.

So, let's assume that somehow you acquire a real-world fair coin. There is one last requirement to be able to "simulate" probability: an infinite number of experiments. Because that is how you interpret probability: Let $a$ be a sequence that is defined as such: $a_{i} = h/i$ , where $h$ is the number of heads so far and $i$ is the number of experiments. If the coin in this experiment is fair, thus the probability of heads $0.5$, this sequence converges to $0.5$.

So, as any other sequence, you can interpet this as follows: After a constant number of experiments, the ratio of heads to experiments will be in the "neighborhood" (i.e. very close) to 0.5. The more experiments you conduct, the smaller this neighborhood will be.

-
will sequence a 'converge' or 'average' to 0.5? – Amit L Apr 5 '11 at 16:46
Converging captures both "averaging" and "approaching from one side" , whether points of the sequence will be on either side of a 0.5 line when plotted depends on the previous tosses. The neighborhood notion covers both cases, see the 2nd formal definition in en.wikipedia.org/wiki/Limit_of_a_sequence – chazisop Apr 5 '11 at 16:56
@Amit L: The sequence will converge (with probability $1$) to $1/2$. – Shai Covo Apr 5 '11 at 16:56
@chasisop: about your comment about building a real world fair coin, it really depends on what you mean by fair and coin. For all practical purposes, any coin you pick up, is fair. Also, it is possible to simulate a coin by a pack of playing cards. (if red card is picked that is heads, etc. – picakhu Apr 5 '11 at 16:58
I mean bulding a coin that is fair to any level of precision, not just for practical purposes. Real world coins are far from fair, due to the anaglyphs on them and the use of several materials. There is a very easy way to simulate a fair coin in reality, but it requires 2 tosses of the same coin. If @Amit L is interested, I can add the construction in my answer. – chazisop Apr 5 '11 at 17:14