What determines if two events are equally likely?

Question

In statistics students are often taught a "naive" definition of probability $$\frac{\# \text{of favorable outcomes}}{\# \text{of possible outcomes}}$$ and a somewhat more rigorous definition of probability: $$P(\bigcup_{j=1}^{\infty}A_j)=\sum_{j=1}^{\infty}P(A_j)$$ where $A_1, A_2$ is disjoint, etc. The general difference between the two is that in the former $P(A_j)$ is assumed to be the same for all $A$, while in the latter they can be different so long as it adds up to 1.

Now a simple scenario where the "naive" definition applies is when you have a fair coin. That is, by definition if the coin is fair, the probability of a single event is 50%. A simple example where the non-"naive" definition applies is when you have an unfair coin, that is heads can occur more frequently than tails. Of course there has been an experiment conducted in the real world that claims there is a 1% bias in tossing a coin for 250,000 trials. At this point it's often brought up that if you toss a coin for an infinite amount of times it should come out to about 50%.

But then how would you claim that one coin is fair and another is unfair? Given an arbitrary amount of trials you could argue either case. And if you get heads 10,000 times in a row, the probability for a fair coin to be head or tails is still 50% though surely most people would claim the probability of tails is 100% given no other prior knowledge of the experiment. I know my thinking is fallacious somewhere, but I can't figure out the problem.

Answer 1

It is just like any problem in statistical inference. You set up the null hypothesis - the coin is fair - and the alternative hypothesis - the coin is unfair. Basically you are doing inference on the parameter $p$ of a Bernoulli distributed variable. It is the hypothesis $p = 0.5$ against $p \neq 0.5$. Then you can derive confidence intervals and so on. Obviously you can never say for certain what the value of $p$ is.

Edit: I think your confusion lies in the conflation between statistical inference and the concept of frequentist probability. In frequentist probability, probability itself is defined as the long-run ratio between the target outcomes and all possible outcomes. It doesn't matter whether the actual probability is accessible to you or not in any particular case. It's just the definition of Probability itself. To try to find estimates of the actual probability in specific cases is a different problem - that is statistical inference, where you might use, for example, the ratio of certain outcomes over other outcomes as estimate of the probability.

You can also look up Bayesian probability, which is a different definition of probability than the frequentist one.

Answer 2

Actually, when you are trying to determine how the coin behaves (its probability distribution) you cannot know for sure which is the correct answer, but you must take a decision in order to continue.

For example, if you have a coin of which you know that either is a fair coin with heads and tails and 50% probability of both, or either is a coin that always tails; and you have to decide how is the coin by tossing it only once. How would you proceed?

Well, let's suppose that you toss the coin and heads. Then, you can be sure that your coin is a fair one. But, if it tails, you cannot know for sure. But the thing is, it is more likely that your coin always tails, if the result obtained is tails. Am I explaining myself?

If you have a coin, and you toss it 10,000 times, and you get 10,000 heads, it actually can be a fair coin, but the more coherent decision according to the result of the experiment is to assume that your coin always heads. You will never get to know the exact way your coin behaves.

---

Actually, you can never be sure that the coin is fair. Suppose you have a coin, and you don't know how it behaves. You tossed it 100 times, and you get 60 heads. Then, you can assume that the coin is unfair and has 60:40. But it could actually be a fair coin, yes. Or it could be an unfair coin with 55:45... You can't be completely sure about it. Of course, the more times you tossed it, the more accuracy you get. But that's it, you can never determine the way the coin behaves.

Hope this is any better.