-1
$\begingroup$

After taking a self-learning course in Probability, I wanted to learn Statistics by myself. I thought one of the best practical ways I could learn if I take a coin and apply statistics to figure out the probability of getting a head. With this, I ask if you could outline the steps I should follow to obtain a number say P(H) is 0.3 or 0.8 etc. It would be useful to mention related topics/theories so that I can read them before I take up this practical experiment.

I was thinking something like this -- repeat tossing for some N times and then use frequency to come to a number for P(H). However, we might need some further theory to decide the exact value of N for which the result would be within say 95% confidence level.

$\endgroup$
7
  • $\begingroup$ I'd have suggested just tossing the coin a load of times and averaging it out - this is the maximum likelihood estimator for the probability of heads. What do you mean by "end-to-end"? $\endgroup$
    – John Doe
    Commented Apr 17, 2018 at 14:14
  • $\begingroup$ What exactly do you mean by "evaluate it"? Do you mean to compute the probability of a certain event about this coin? Or do you mean to use hypothesis testing to see if it is actually biased and how biased? $\endgroup$ Commented Apr 17, 2018 at 14:14
  • $\begingroup$ @ John Doe - When I googled, found some blogs but the answer were not complete. So proactively I write end-to-end to indicate that i'm looking for a complete answer following which I would be able figure out what is the P(H) for a given coin. Btw, tossing infinitely many times would a great way to do it..but like to know what would be more realistic approach $\endgroup$
    – KGhatak
    Commented Apr 17, 2018 at 14:40
  • $\begingroup$ @ glowstonetrees - By evaluation I mean to come up with a number say 0.3 (how biased) if possible. $\endgroup$
    – KGhatak
    Commented Apr 17, 2018 at 14:41
  • 1
    $\begingroup$ I am not the downvoter, but I assume that whoever is the same as the person who has put a close vote on. It is for the following reason: This question is missing context or other details: Please improve the question by providing additional context, which ideally includes your thoughts on the problem and any attempts you have made to solve it. This information helps others identify where you have difficulties and helps them write answers appropriate to your experience level. $\endgroup$
    – John Doe
    Commented Apr 17, 2018 at 20:47

1 Answer 1

2
$\begingroup$

I'd say that, by the definition of a coin being biased, you're assigning each outcome a certain probability. We denote the probability of (the union of) all possible events to be 1, and then we pick a certain event out of all the possibilities. The union of all possible events is called the sample space (denote it $\mathcal{S}$, for instance). It contains all possible outcomes of your "experiment".

In this case, we have two possiblities: heads or tails. So we can write the sample space as: $\mathcal{S} = \{H, \; T\}$, where $H$ means "heads" and $T$ means "tails".

Now, the probability that anything within a sample space occurs is 1. We write this as:

$$\operatorname{P} (\mathcal{S}) = 1$$

In this case, we can also write $\mathcal{S}$ as a union of the two events that compose it:

$$\mathcal{S} = H \cup T$$

So:

$$\operatorname{P} (\mathcal{S}) = \operatorname{P} (H \cup T) = 1$$

These two elements are disjoint, which means that they can't happen at the same time. This means that the chances of $H$ or $T$ happening are the same as the chances of $H$ happening plus the chances of $T$ happening. This is a bit of a difficult concept to wrap around your head, especially in more complex cases. Hence,

$$\operatorname{P} (H \cup T) = \operatorname{P} (H) + \operatorname{P} (T) = 1$$

For an unbiased coin, $\operatorname{P} (H) = \operatorname{P} (T)$, which would make:

$$\operatorname{P} (H) + \operatorname{P} (T) = \operatorname{P} (H) + \operatorname{P} (H) = 2\operatorname{P} (H) = 1$$

$$\Rightarrow \operatorname{P} (H) = 0.5$$

However, for a biased coin, by definition $\operatorname{P} (H) \not = \operatorname{P} (T)$, which means that:

$$\operatorname{P} (H) = 1 - \operatorname{P} (T)$$

I hope this clarifies your understanding of the matter.

EDIT:

Now I finally understand what you mean. What you need in this case is testing, or proving, with a certain confidence, that the coin is biased or not.

We start with the alternative hypothesis: "The coin is biased", and then the null hypothesis is the opposite. To show that the coin is biased, we will try to show (with a certain confidence) that it is not unbiased. Hence, we're trying to disprove the opposite of the alternative hypothesis, the null hypothesis: "The coin is not biased."

$$H_a: \text{The coin is biased.}$$ $$H_0: \text{The coin is not biased.}$$

To "disprove" $H_0$ we calculate the probability of the actual outcome we observed, assuming that the coin is unbiased. Obviously, you'd require some observations. I'll provide some as an example.

So suppose we observed that, out of 100 coin tosses, 57 were heads and 43 were tails. Now, we assume that each toss is independent and that the probability of each toss does not change throughout the experiment. These are the characteristics of a Binomial distribution.

Now we can see that $H_0$ is equivalent to saying that $X$, the number of observed heads, is distributed $\sim \operatorname{BIN}(100, p = 0.5)$

So, because by our question of whether the coin is biased, we're asking whether heads has a higher chance of occurring than tails, we calculate

$$\begin{align} \operatorname{P}(X \ge 57) & = \operatorname{P}(X = 57) + \operatorname{P}(X = 58) + \cdots + \operatorname{P}(X = 100) \\ & = \binom{100}{57} (0.5)^{57} (0.5)^{43} + \binom{100}{58} (0.5)^{58} (0.5)^{42} + \cdots + \binom{100}{100} (0.5)^100 (0.5)^0 \end{align}$$

Typically, we'd approximate this probability with the Normal distribution, but modern computers can handle calculating that (you can check the sum here). So

$$\operatorname{P}(X \ge 57) \approx 0.09667$$

Now, depending on your confidence interval, you go on to concluding whether you can reject $H_0$ and therefore accept $H_a$ or whether you cannot reject $H_0$ and therefore cannot accept $H_a$.

A popular choice for a confidence interval is 0.05, but there is a lot of debate on whether it should be that popular or not (some studies show that there is a lot of space for the so-called p-value hacking for $p = 0.05$, but this is beyond the question). For some tests you'll need a much smaller confidence interval (aka it is more difficult to reject $H_0$; i.e. medical illness detection procedures), while for others you can be much more lenient (i.e. tossing a coin).

In this case if your confidence level is at 0.10, because 0.09667 < 0.10, you can reject $H_0$ and accept $H_a$, but if it were at 0.05, you wouldn't be able to do this.

Basically, you're calculating the probability of the outcome you observe if the distribution is binomial with $n = 100$ and $p = 0.5$. If this probability is high, it means that it is likely that the distribution is appropriate for what you are measuring, and therefore the coin is not biased as $\operatorname{P}(H) = \operatorname{P}(T) = p = 0.5$.

However, if such probability is low, it means that it is not likely that what you're observing follows the binomial distribution. You can then go on to conjecture why this is: maybe the coin is biased ( $\operatorname{P}(H) \not = \operatorname{P}(T)$ ), maybe the throws are not independent, maybe the probability changes from throw to throw...

And you're done.

$\endgroup$
7
  • 1
    $\begingroup$ @ Alex Lostado - My question is a bit different. If I hand you a coin what you'll do to tell me the exact (may be with some caveat added e.g. confidence level) value of P(H) - this is my question. $\endgroup$
    – KGhatak
    Commented Apr 17, 2018 at 16:07
  • $\begingroup$ @KGhatak well, it is theoretically possible to measure its mass and shape and mass distribution to find if there's a heavier side, or anything else that could affect the toss of the coin, and then calculate how this asymmetry affects the probability of heads or tails. This is very difficult in practice, so the best thing would be to "test" it using the null and alternative hypotheses. $\endgroup$ Commented Apr 17, 2018 at 16:15
  • 1
    $\begingroup$ @ Alex Lostado - Can't we assume that the physics of tossing doesn't induce any variability!! I was thinking something like, repeat tossing for some N times and then use frequency to come to a number for P(H). However, we might need some theory (may be the law of large numbers - I've little knowledge on this though) to decide the exact value of N and it would be essentially a function of some confidence level. In essence, the final answer would be like if we toss for N=1M then we'll get an answer with 80% confidence and if N=1B, then confidence level would be 95%. Wouldn't that be possible! $\endgroup$
    – KGhatak
    Commented Apr 17, 2018 at 16:30
  • 1
    $\begingroup$ Sorry if I misled you. But I'm looking for an exact number say P(H)=0.57 or 0.3 or so. Not just whether it's biased or not. $\endgroup$
    – KGhatak
    Commented Apr 17, 2018 at 17:40
  • 1
    $\begingroup$ I think I've got a good starting point here. Thank you $\endgroup$
    – KGhatak
    Commented Apr 17, 2018 at 18:56

Not the answer you're looking for? Browse other questions tagged .