What questions can Probability answer which Statistics can't?

Question

I was wondering, why we need to study Probability. I have come to the following personal conclusion:

A lot of things take place in our everyday life, or we get involved in a lot of things where we can not predict anything for certain beforehand, but we can conclude some forecasts. For instance, we can not answer the following questions for certain, but we can give some forecasts:

1. What would be the share price like for the company 'X' in the next month?
2. How fast or how big the economy of a country 'Y' would grow in the next quarter?
3. How likely is a region to be hit by earthquake next month?
4. ... ... ...

Again, there are some problems in our life where the incident depends on pure luck, but, we have to come to a definite conclusion. For instance, we need to answer the following question in out daily lives:

1. Who would be declared a winner if a Cricket match is abandoned half way through?
2. How would the batting target be like If a Cricket match is interrupted by rain and wasted almost half an hour in the process?
3. How would two players share the bet-money if a gambling match is abandoned half way through?
4. ... ...

Now, the question arises, why isn't inferencial statistics good enough to answer these questions?

Well, the answer is, in these cases, either we don't have sufficient data in our hands to be analyzed (e.g. gambling match), or, analyzing a large data doesn't help much (e.g. even though we may have a large amount of data for a company to be analyzed, a share price can drop or rise at any time for arbitrary reasons).

Is my understanding correct?

If my understanding is correct, why do we analyze weather data using probability and Random Processes, as weather forecasts do not change much on yearly basis?

Answer 1

The simple answer is that Inferential Statistics simply cannot exist without probability.

Every major result in Inferential Statistics has a rigorous underpinning in Probability/measure theory.

The Laws of Large Numbers say obvious things: "The sample mean will converge in probability/almost surely to the true population mean", but how on earth would you prove this without formal probability axioms?

But then go further, what about the completely counter intuitive results like the central limit theorem. Why on earth should we expect the sample mean and sample variance to be asymptotically normal?

Finally, consider Bayesian statistics. How could you possibly undertake Bayesian inference without a proper understanding of conditional probability?

On the surface inferential statistics may seem to be common sense results. However, a good chunk of those common sense results require extensive and rigorous proofs, which is where probability theory is important. Without probability theory, statistical inference would not have any of the important results used every day.

Answer 2

Probability and statistics are related areas of mathematics which concern themselves with analyzing the relative frequency of events. Still, there are fundamental differences in the way they see the world:

Probability deals with predicting the likelihood of future events, while statistics involves the analysis of the frequency of past events. Probability is primarily a theoretical branch of mathematics, which studies the consequences of mathematical definitions.

Statistics is primarily an applied branch of mathematics, which tries to make sense of observations in the real world. Both subjects are important, relevant, and useful. But they are different, and understanding the distinction is crucial in properly interpreting the relevance of mathematical evidence.

This distinction will perhaps become clearer if we trace the thought process of a mathematician encountering her first craps game:

If this mathematician were a probabilist, she would see the dice and think :

Six-sided dice? Presumably each face of the dice is equally likely to land face up. Now assuming that each face comes up with probability 1/6, I can figure out what my chances of crapping out are.

If instead a statistician wandered by, she would see the dice and think

Those dice may look OK, but how do I know that they are not loaded? I'll watch a while, and keep track of how often each number comes up. Then I can decide if my observations are consistent with the assumption of equal-probability faces. Once I'm confident enough that the dice are fair, I'll call a probabilist to tell me how to play.

In summary, probability theory enables us to find the consequences of a given ideal world, while statistical theory enables us to to measure the extent to which our world is ideal.

Quoting @Xiaomi's answer on this: Inferential Statistics simply cannot exist without probability. Every major result in Inferential Statistics has a rigorous underpinning in Probability/measure theory.

This link can be of help.

Answer 3

I'm no expert, but here's my guess.

You write:

Now, the question arises, why isn't inferencial statistics good enough to answer these questions?

My guess is that it is good enough.

But, of course, there's good ideas in inferential statistics, and bad ideas. How do we keep the good ones, while throwing out the bad ones?

If the t-test formulae were a bit different... would that be okay? If $p$ values were $2 \times (\mbox{what they currently are}),$ would that be okay? Or would it be a problem? What if they were defined differently, so that for more than 100 data points they were smaller than they currently are, and for less than 100 data points, they were more than they currently are. Would that be okay?

The only way to answer this question is to use ideas from theoretical statistics.

But since theoretical statistics has probability theory as its foundation, you really can't get away from probability theory.