# Sources to learn and understand advanced probability in ML models

Could someone please suggest some good positions on probability (and perhaps statistics)?

My ultimate goal would be to learn and understand Machine Learning and its models, such as Neural Network, Support Vector Machine, Hidden-Markov Model etc. The mathematics behind it tend to be very advanced.

I found a couple of positions: an online book for free The Elements of Statistical Learning: Data Mining, Inference, and Prediction. and two papers just on HMM: What can HMMs Can Do or A tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. While there are detailed explanations of theory, I personally find them really hard to get through.

Do you think there is a good book, online sources, etc. I can start with? I am absolutely aware of the fact than there might be more than just one book etc. I think I need to start with basics of probability in practice, then move to random variables, joint probabilities and then, independence, Bayesian Theorem, chain probability rule, etc.

I understand the topics are advanced but and without a proper understanding of probability I am afraid I cannot go much far.

Thanks.

EDIT: I also found another book for beginners: Understanding Probability by Henk Tijms on Amazon. Some strongly suggest this book but there are a number of others who don't recommended it claiming the book is not well organised, e.g.

"link from the binomial distribution to the Poisson distribution is being derived on a few lines. If the book wants you to understand probability these very important relations between distributions should be explained in much more depth, in my view"

or

"The author's love and enthusiasm for the subject shows on every page. However, my students found the book incomprehensible. When I polled my class of 65 students after 8 weeks, I was shocked to discover that all 65 of them preferred to have no textbook at all than to have this one.". Frankly, those comments put me off.

Is there anyone who have after course of probability and have apportunity to learn from this book?

• what's your background? Oct 8, 2015 at 11:30
• undergraduate engineering, if it's what you mean. So I have studied, calculus, algebra, etc. A little bit of matrix algebra but we have not covered matrix calculus Oct 8, 2015 at 11:50

For you Probability and Random processes by Grimmet seems to be a good starting place. Along with that you can read these lecture notes by Bruce Hajek.

Once you've got reasonable hold on these books you can go for studying some measure theory since without it, it is quite impossible to learn advance graduate level probability which you might be requiring in your further exploration into Machine learning based research. For measure theory there are many good resources. One very elementary book is by Schilling which introduces the subject at first hand. Along with this then you could probably go for a graduate level probability book like Probability theory by Rick Durrett.

Of course, I should mention the Machine learning book named Pattern recognition by Christopher Bishop which is great in explaining difficult machine learning concepts quite clearly. Once you have mastered these, you can confidently look at advanced research level Machine Learning literature with sufficient ease.

Addendum: I forgot to add these two very good books that can serve as probability theory references besides Grimmet, that is if you find it a little difficult. They are the following: Introduction to the theory of statistics by Mood Graybill and Boes, and Introduction to probability Models by Sheldon M. Ross

• Thanks for this Samrat. this is really good answer :) You gave me a new key word "Measure theory". I will try to look for enything and see if I can follow. Ty. Oct 8, 2015 at 12:23
• Samrat, if I may ask, have you got opportunity to read the book I have mentioned in editted part of my question? it is said that Grimmet's book is going fast may be hard to understand for beginners. Do you think the notes you have provided link to are good complementary for the Gimmet's book? Thanks Oct 15, 2015 at 8:59
• @Celdor, I just glimpsed through the book in your edit and I think I will go with the people not recommending this book. Some of the chapters seem not written in a way that can be grasped easily. I am adding a few more references in my answer that I forgot to add before. Oct 15, 2015 at 10:35
• I need textbook with examples. I'll try to have a go with "Probability and Random processes". There's also a book "Thousand examples ..." of the same author. If I find them hard, I'll try the other books u have suggested. In the end, I still have goole and youtube, and here. So if I stuck I'll ask for help. Thanks. Oct 23, 2015 at 9:34
• You are welcome @Celdor. Oct 24, 2015 at 4:59