3
$\begingroup$

We have run an experiment in which some good chap has sat down and flipped a coin 100 times. At the end of the 100 flips he has tallied 40 Heads and 60 Tails. Now this seems like something is up with the coin. The question is whether or not this coin is biased.

I have already determined that the mean p=1/2 and the standard deviation is 5. If we take the mean as a random variable of a normal distribution about the mean, then the experimental results we obtained are 2sigma from the mean. My first question is what does it mean if the results are outside one sigma?

Next I proceeded to find the probability that the p-value of the getting a heads was instead 4/10. I used the function 100C40 *p^40 *(1-p)^60 and integrated this (dp) from 0.35 to 0.45. My result was 0.006. Now does this mean that the probability of getting a p is between 0.35 and 0.45 is 0.006 ? But I feel in this method I should be comparing this p against something.

I suppose my problem really lies in interpreting the results and their meaning.

$\endgroup$

3 Answers 3

Reset to default
1
$\begingroup$

p-value: The probability of obtaining the test statistic, if the null hypothesis is true.

There is a write up regarding using hypothesis testing and p-values in testing to see if a coin is fair here: http://en.wikipedia.org/wiki/P-value#Interpretation

Unfortunately, there is no standardized p-value level to reject a null hypothesis. Depending on which field you are in, the rejection levels can be p < 0.05 (5% significance level) to p < 0.025 (2.5% significance level). Or even P < 0.10 (10%).

Basically, if you reject because p < 0.10, you are saying that if the null hypothesis is true, the probability of the data occurring is less than 10%. You feel that the null is unlikely to be true.

This is of course subjective.

P.S.

Note that there are no "acceptance" levels. While we may reject or not reject the null hypothesis. We do not say we accept it.

$\endgroup$
1
$\begingroup$

The question is impossible to answer, because an important information is missing.

The missing information is: What is the probability that a random coin is biased? More precisely, what is the distribution of biased coins with different biases in the initial population of the coins? What is the prior probability that the coin was biased, before you run the experiment?

To see why this information is relevant, imagine that the experiment was repeated in two different countries. In country A, biased coins simply do not exist. In country B, only 1/3 of coins is fair, 1/3 is head-biased, and 1/3 is tail-biased. In both countries the experimenters run the experiment, in both countries they receive the same 40:60 outcome. But because of the difference between the countries, the two experimenters must interpret their results differently.

In country A, where biased coins do not exist, we must reason according to Sherlock Holmes that "when you have eliminated the impossible, whatever remains, however improbable, must be the truth". Biased coins are impossible, therefore the truth must be that a very improbable event happened with a fair coin.

In country B the initial odds of fair coin versus tail-biased coin are 1:1, so after the experiment, the resulting odds are something like 1:50. (I am not sure about the exact number. Also it depends on how strongly tail-biased those tail-biased coins are. If they are so biased that they provide tails in 90% of flips, then receiving a 40:60 outcome with such coin is even less probable that receiving a 40:60 outcome with a fair coin.)

In what country did you do your experiment? I guess it's somewhere in between the example A and B countries: there are biased coins, but they don't make 1/3 of the coin-population. But you have to make an estimate. Do you think that 1 coin in 10 is biased? Or 1 in 1000?

  • Calculate a probability that you have achieved given result with a fair coin. Probability that a random coin in coin-population is fair × probability that flipping a fair coin 100 times will give you 40:60 outcome.

  • Calculate a probability that you have achieved given result with a tail-biased coin. Probability that a random coin in coin-population is tail-biased × probability that flipping a tail-biased coin 100 times will give you 40:60 outcome. (You may do this calculation multiple times, for different degrees of bias.)

  • The ratio of these results is the ratio of posterior probabilities of having started with a fair coin vs having started with a tail-biased coin.

(For a more complete calculation you should also include probabilities that you made a mistake in calculation, or that the person who flipped the coins has a skill to make a fair coin land on given side with higher probability. Sometimes these probability are not insignificant compared with probabilities of achieving the result by regular methods.)

Without this data you can't calculate the real probability, because you don't have the relevant inputs. You can only calculate "something", which may be called a traditional interpretation, but is incorrect. But it's traditional, which means that many people make this mistake. (Achieving an incorrect but credible result may be preferable to a correct calculation containing too many unknown values.) The correct method of calculating probabilities is called "Bayesian".

$\endgroup$
7
  • $\begingroup$ To clarify: The incorrect traditional answer starts with an implicit assumption that the fair coin and the biased-exactly-as-you-need-it coin have prior odds 1:1. It's easy to not notice this assumption, because multiplication by 1 is invisible; I guess most people really don't notice it. -- But if you think about this assumption, you will probably agree that it is not realistic. In reality, we don't have 50% fair coins, and 50% tail-biased coins. So why should we pretend that in our equations? $\endgroup$
    – Viliam Búr
    Oct 10, 2012 at 9:54
  • 1
    $\begingroup$ Nice write, I enjoyed reading it. But in this case, the statistical test is set up to handle both country A and B. Regardless of the characteristic of the coin population, the interpretation is the same: the p-value is the probability of obtaining the experimental data given that the null hypothesis (the coin is a 1:1 fair coin) is true. $\endgroup$
    – Legendre
    Oct 10, 2012 at 10:23
  • $\begingroup$ Exactly. P-value is probability of getting experimental data, given the null hypothesis. And the original question was "whether or not this coin is biased", that is probability of the null hypothesis, given experimental data. - We should reject the hypothesis because the hypothesis is unlikely. Not because the experimental data given the hypothesis is unlikely. (Maybe the data is very unlikely both given our hypothesis and given the null hypothesis. Maybe the data is likely both given our hypothesis and given the null hypothesis.) We can't reject the hypothesis based on p-value only. $\endgroup$
    – Viliam Búr
    Oct 10, 2012 at 10:54
  • $\begingroup$ In the usual statistical hypothesis testing framework (en.wikipedia.org/wiki/Statistical_hypothesis_testing), we reject the null when "the data is unlikely given that the null is true". The null is said to be "rejected" under this framework. Externally, the scientist is still free to consider it as a valid hypothesis or supplement his results with other evidence. Although I agree that "rejection" in a statistical hypothesis testing framework is often confused with actual rejection, which shouldn't just rely on this kind of testing alone. $\endgroup$
    – Legendre
    Oct 10, 2012 at 12:51
  • $\begingroup$ So, in absence of more data, neither "rejecting the null hypothesis" nor "not rejecting the null hypothesis" provides an answer to a question whether the coin is more likely to be fair or more likely to be biased. $\endgroup$
    – Viliam Búr
    Oct 10, 2012 at 14:26
1
$\begingroup$

If the outcome of each toss is independent of the previous tosses, and if the probability ($p$) for Heads remains the same, then the number of Heads during 100 tosses will be a Binomial distributed random variable $\text{Bin}(100,p)\ .$ Using this distribution you can expect to obtain $100p$ ''number'' of Heads; this is the expected value of the Binomial distribution, which may or may not be an integer.

If the coin is unbiased then $p=0.5$ so you would expect 50 Heads among 100 tosses. The chap doing the tossing tallied 40 Heads, so the question becomes Is 40 sufficiently far apart from 50 for you to claim that the data indicates the coin to be biased?

If the coin is indeed unbiased, how likely is it to obtain 40 Heads (or less)? (This is the p-value of the gathered data.) If the probability is high (traditionally, more than 0.05) then the conclusion would be that 40 is not sufficiently far away from 50 to be statistically significant; the gathered data does not support the claim that the coin is biased.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .