Sources to learn and understand advanced probability in ML models

Question

Could someone please suggest some good positions on probability (and perhaps statistics)?

My ultimate goal would be to learn and understand Machine Learning and its models, such as Neural Network, Support Vector Machine, Hidden-Markov Model etc. The mathematics behind it tend to be very advanced.

I found a couple of positions: an online book for free The Elements of Statistical Learning: Data Mining, Inference, and Prediction. and two papers just on HMM: What can HMMs Can Do or A tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. While there are detailed explanations of theory, I personally find them really hard to get through.

Do you think there is a good book, online sources, etc. I can start with? I am absolutely aware of the fact than there might be more than just one book etc. I think I need to start with basics of probability in practice, then move to random variables, joint probabilities and then, independence, Bayesian Theorem, chain probability rule, etc.

I understand the topics are advanced but and without a proper understanding of probability I am afraid I cannot go much far.

Thanks.

EDIT: I also found another book for beginners: Understanding Probability by Henk Tijms on Amazon. Some strongly suggest this book but there are a number of others who don't recommended it claiming the book is not well organised, e.g.

"link from the binomial distribution to the Poisson distribution is being derived on a few lines. If the book wants you to understand probability these very important relations between distributions should be explained in much more depth, in my view"

or

"The author's love and enthusiasm for the subject shows on every page. However, my students found the book incomprehensible. When I polled my class of 65 students after 8 weeks, I was shocked to discover that all 65 of them preferred to have no textbook at all than to have this one.". Frankly, those comments put me off.

Is there anyone who have after course of probability and have apportunity to learn from this book?

Answer 1

For you Probability and Random processes by Grimmet seems to be a good starting place. Along with that you can read these lecture notes by Bruce Hajek.

Once you've got reasonable hold on these books you can go for studying some measure theory since without it, it is quite impossible to learn advance graduate level probability which you might be requiring in your further exploration into Machine learning based research. For measure theory there are many good resources. One very elementary book is by Schilling which introduces the subject at first hand. Along with this then you could probably go for a graduate level probability book like Probability theory by Rick Durrett.

Of course, I should mention the Machine learning book named Pattern recognition by Christopher Bishop which is great in explaining difficult machine learning concepts quite clearly. Once you have mastered these, you can confidently look at advanced research level Machine Learning literature with sufficient ease.

Addendum: I forgot to add these two very good books that can serve as probability theory references besides Grimmet, that is if you find it a little difficult. They are the following: Introduction to the theory of statistics by Mood Graybill and Boes, and Introduction to probability Models by Sheldon M. Ross