5
$\begingroup$

I am an Engineering Graduate (with a strong background in Probability/Measure Theory, Linear Algebra and Calculus) wanting to dig deep into Deep Learning and Neural Networks, and I'm looking for mathematically thorough and rigurous (almost pedantic) material and books to go through the material, but I don't seem to find something of my liking among all the myriad of hype-train books out there considering how popular Machine Learning has become in the last years. Even the rather non-hands-on books about it seem to be too hands-on to me.

I have both Pattern Recognition and Machine Learning by Bishop, which I started working through (about 150 Pages), as well as Deep Learning by Goodfellow (which I only skimmed through), but I am not happy with either. They both seem to try to cover too much material, with little rigorousness and detail, getting lost in specificalities. I am only speaking out of intuition here, but I feel like due to the assumption that the average reader lacks real depth in the areas of Probability, Linear Algebra and Calculus (which is clear by the sloppiness of the use of them), I feel like there's the need for too much explanation.

What I'm looking for is a more concise, but more mathematically precise book on the fundamentals of Deep Learning / Machine Learning that assumes a strong and formal knowledge of Linear Algebra, Probability Theory (Kolmogorov, Probability Spaces, etc.), and Calculus where I can draw my own (mathematical) conclusions and without too much specific gibberish (e.g. I don't need 5 examples, graphs and intuitive explanations on the difference between bias and variance, I just need a rigorous definition of both, and a few important Theorems around them which I can work with).

My reasoning here is that the field is so complex already and expanding in an exponential manner on so many directions, and understanding concepts like regularization and optimization seems so daunting, because of the lack of mathematical abstractions behind it, so it seems that every application needs its own interpretation. As experience has showed me, this happens when there's a lack of fundamental mathematical knowledge that can help extrapolate into other examples without the explicit need to understand the details of it.

An analogous example is say with systems of differential equations: You might need to understand a few examples (e.g. Romeo and Juliet/Love affairs https://ai.stanford.edu/~rajatr/articles/SS_love_dEq.pdf) but then it suffices to be able to understand those systems as general mathematical constructs and how to solve them/approach them without having to understand intuitively every single example a priori.

Thank you

$\endgroup$
3
  • 1
    $\begingroup$ If you want to use a lot of analysis and even higher algebraic constructs, the only intersection with machine learning I've seen comes from equivariant neural nets. Check out this Review: link.springer.com/article/10.1007/s10462-023-10502-7. $\endgroup$
    – user1058535
    Commented Apr 10 at 7:37
  • 1
    $\begingroup$ The math behind deep learning isn’t very deep. I wonder if you’d like to glance at my notes on logistic regression, mainly section 8. Deep learning isn’t much more complicated than that, except that the prediction function gets fancier. $\endgroup$
    – littleO
    Commented Apr 10 at 7:41
  • 1
    $\begingroup$ Do you know of this? $\endgroup$
    – J.G.
    Commented Apr 10 at 7:51

1 Answer 1

2
$\begingroup$

Don't know if you will find this interesting but it might be worthwhile giving Kevin Murphy's book(s) on probabilistic machine learning a try.

https://probml.github.io/pml-book/book1.html

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .