0
$\begingroup$

I'm an engineering student who has taken one undergraduate course in probability theory, but that's all my exposure so far. I'm trying to get into machine learning and need to develop more of a background for this purpose.

I hear about different types of probability theory and where it builds from, but the connections between these are extremely hazy to me. For example, I've heard of probability theory being grounded on top of measure theory, but have also heard of it based on logic, have heard the terms Kolmogorov and de Finetti probability theory, but can't make sense of what's going on here. Is there different factions of probability theorists who disagree with each other or do they all play a part in some coherent framework? Would anyone give me a beginner's synopsis of the relationships and layout between these things?

$\endgroup$

closed as primarily opinion-based by Adam Hughes, Shailesh, projectilemotion, C. Falcon, Jack Feb 11 '17 at 3:09

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    $\begingroup$ I think "Kolmogorov probability theory" means the measure theory framework. You don't need any of that to have an understanding of machine learning. What you really need is "naive" probability and statistics using elementary calculus. $\endgroup$ – mathematician Feb 10 '17 at 22:38
  • $\begingroup$ @mathematician I picked this book up by Jaynes called probability theory the logic of science and was also wondering if this is compatible or incompatible with measure theory? $\endgroup$ – Austin Feb 10 '17 at 22:39
  • 2
    $\begingroup$ @Jake It seems that Jaynes' book is introduction to probability + mathematical statistics. The Measure theoretic framework is mentioned only in the appendix. $\endgroup$ – V. Vancak Feb 10 '17 at 22:45
  • 1
    $\begingroup$ Most people dealing with probability foundations are using measure theory as their foundation. A basic philosophical tenet of the subject is that it should not really matter what your foundation actually is, pretty much everything should go through the same way. A merit of the measure theoretic approach is that you need an extremely small axiomatic system to get off the ground. $\endgroup$ – Ian Feb 10 '17 at 22:53
  • 1
    $\begingroup$ A drawback of the measure theoretic approach is that the measure spaces studied in probability theory are for the most part not "concrete": in other words, their elements don't really mean anything. This makes the whole proceeding have some additional, arguably unnecessary, abstraction. Still, probability theory went through a veritable renaissance upon the widespread adoption of the Kolmogorov axioms in the early 20th century. $\endgroup$ – Ian Feb 10 '17 at 22:56
1
$\begingroup$

Kolmogorov probability theory defines the probability measure on $\sigma$-algebras, and the measure is countable additive. This is what pretty much everybody uses.

De Finetti defines the the probability measure on algebras, and the measure is only finitely additive. It is an alternative theory, but beyond you would actually need.

My suggestion would be to start with basic books, such as Sheldon and Ross, and upgrade to more advanced things (Grimmet is good for engineering students, Billingsley is good too) only after you have become confortable with conditioning, independence, joint pdf's,etc.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.