# Does set theory help understand machine learning or make new machine learning algorithms?

When I was in a university, I didn't major in math but took some math classes. However, I dropped out of math classes pretty quick.

Some person recommended that I learn some set theory because it'll help me with Machine Learning . He recommended a book named Naive Set Theory by Halmos

Other people say it's not going to help much because set theory lives on a far higher abstraction level than mathematics used in machine learning do.

Since I don't know math well, I can't judge who's right.

Can anyone tell me how set theory is going to help me with machine learning ?

• Others have given good answers to your question but one thing I would like to point out is that Naive Set Theory is a super short book. You could reasonably get through it in a weekend and I am sure that you would gain quite a bit of helpful knowledge. – Budge Apr 14 '16 at 1:34

TL;DR : Set theory is a good thing to know , but there is more math out there which is actually critical for Machine Learning

You should probably learn Math in this order :

1) Pick up this OR this and flirt with the sections on proofs , set theory and its notations , functions , combinatorics , graphs ; till the time you feel comfortable solving a simple non-trivial problem . Do not pick up books like Halmos , please , -- not discrediting the book in any way - just not required here .

2) Learn Single Variable Calculus from here and here . Once you understand the intuition behind it , pick up a standard textbook and solve a lot of problems .

3) Learn a bit of Multi-Variable Calculus from here

4) Learn Linear Algebra from Strang . Ah ! the joys of learning from Strang . Seriously . You can do yourself no further good . Hop around to other MOOCs like this one .

5) Learn Probability and Statistics , IMO the most important math field required for ML . There are already pretty good resources and answers on the net , so will not repeat.

6) Learn how to apply statistics , probability theory to real-world problems intuitively by learning Data Mining and doing competitions on Kaggle

7) Take up Ng's Wonderful Course and embark on a hard yet fun journey of ML

Hope this helps :)

Disclaimer : I am not a ML Researcher , only a CS undergrad who has studied ML and is now working on a ML-based paper under a professor ; so it is possible that the order might vary according to your mileage

• Hi @Minsky : (1) I do not want you to avoid Set Theory ; get proficient with it , but you really do not need to go deep into it , which Halmos does . (2) The MIT Math Lectures are pretty nice as well the lectures available here . (3) 3 calculus materials because the first one gives you a taste and the intuition , the second and the third are full blown courses . (4) Not heard of "Linear Algebra Done Right" , but Strang is perfect . Any more doubts , will be happy to answer :) If you like my answer - you can accept it by selecting the tick :) – pranav Apr 24 '15 at 9:30
• Thanks for clarification, @pranav . (1) Do you know a better material than Rosen's? Amazon reviews are very very angry toward Rosen. (2) I am not sure if it's a good approach to learn calculus by MIT video lectures instead of a book. How do you compare videos and books? (3) I thought Strang's MIT video lectures are meant to be accompanied by his book. Am I right? – crocket Apr 24 '15 at 12:01
• @Minsky: (1) The difficulty with replacing Rosen is that it's kind of a one-stop shop for a ton of disparate mathematical disciplines, without assuming any background in any of them. However, as a zeroth-order approximation, I would see what you think of Morash's "Bridge to Abstract Mathematics." It only covers proofs-sets-functions; but you might find it a good pace. (2) I think learning from lectures is a very good idea; it took me an extremely long time to learn math from books. Of course, this depends on your style of learning. – Eric Stucky Apr 24 '15 at 12:13
• @pranav I don't necessarily want to read 'how to prove it' from cover to cover unless it's a fast read and if Rosen contains similar topics. – crocket Apr 24 '15 at 12:31
• @pranav It seems to solve my problem for now. I'll come back after reading 'how to solve it' and discrete mathematics. – crocket Apr 25 '15 at 0:21

Machine learning, if you really want to understand it rather than just believe the algorithms do what they are supposed to do, requires quite a heavy dose of mathematics. That mathematics is written in a language and that language is set theory a la Halmos (i.e., the naive kind). If you wish to properly understand the mathematics involved in ML, then it is a good idea to have a good grasp of the underlying universal language which is set theory. Halmos' book is quite short and does not go to the very high level of abstraction that set theory proper goes to. If ML is your goal, I'm would not say Halmos is a necessity, but it is certainly a worthwhile reading. Just don't expect it to translate into any algorithm.

• If I read halmos' book, will I be able to understand mathematics used in ML and make new ML algorithms? Can you refer me to the list of math fields that I need to learn for understanding ML? – crocket Apr 15 '15 at 14:00
• @Minsky: This is pretty unlikely. At best you will be able to be able to read ML algorithms; but you will need more to understand them, and considerably more to be able to crate new ones. – Eric Stucky Apr 24 '15 at 12:16