In statistics students are often taught a "naive" definition of probability $$\frac{\# \text{of favorable outcomes}}{\# \text{of possible outcomes}}$$ and a somewhat more rigorous definition of probability: $$P(\bigcup_{j=1}^{\infty}A_j)=\sum_{j=1}^{\infty}P(A_j)$$ where $A_1, A_2$ is disjoint, etc. The general difference between the two is that in the former $P(A_j)$ is assumed to be the same for all $A$, while in the latter they can be different so long as it adds up to 1.
Now a simple scenario where the "naive" definition applies is when you have a fair coin. That is, by definition if the coin is fair, the probability of a single event is 50%. A simple example where the non-"naive" definition applies is when you have an unfair coin, that is heads can occur more frequently than tails. Of course there has been an experiment conducted in the real world that claims there is a 1% bias in tossing a coin for 250,000 trials. At this point it's often brought up that if you toss a coin for an infinite amount of times it should come out to about 50%.
But then how would you claim that one coin is fair and another is unfair? Given an arbitrary amount of trials you could argue either case. And if you get heads 10,000 times in a row, the probability for a fair coin to be head or tails is still 50% though surely most people would claim the probability of tails is 100% given no other prior knowledge of the experiment. I know my thinking is fallacious somewhere, but I can't figure out the problem.