Good set of exercises with solutions for linear algebra

Question

I'm trying to get a solid knowledge of linear algebra for statistics and machine learning. I didn't study math during college/university. I have the very basic knowledge of what is a dot product, a matrix inverse, and a transpose, but I tend to stumble on concepts like rank of a matrix. There are several books out there to learn linear algebra (to name just a couple):

• Linear Algebra (Dover Books on Mathematics) by Shilov
• Introduction to Applied Linear Algebra – Vectors, Matrices, and Least Squares by Boyd

However, whichever course of stat / math you take starts always with the teacher saying: "Only listening to my class is useless if you don't do the exercises". Alright, but if you are studying on your own, you also need the answers (being confident that your answer to an exercise is the right one when you are actually flatly wrong is astonishingly easy according to my own experience...). The above mentioned books have exercises but don't provide the answers. Could somebody recommend a source of exercises with solutions with the final goal of being able to understand (at least a little better) a reference book of machine learning like The Elements of Statistical Learning by Hastie and Tibshirani?

Answer 1

Give Linear Algebra Problem Book by Paul Halmos a try. It is written in a conversational style and has both hints and solutions at the back of the book.

Another book that has answers to exercises is Jim Hefferon's Linear Algebra, which is freely available online, but I personally have not read it.

Answer 2

Out of my favorite texts on linear algebra, three have solved exercises:

• Jim Hefferon, Linear Algebra.

• Neil Strickland, Linear Algebra for Applications - MAS201.

• Robert Beezer, A First Course in Linear Algebra comes with a solution manual.

The first one is a vector-space-based approach while the second is all about matrices. Both are mathematically rigorous and (for all I have checked) written well. I have no experience with the third. I don't know if they take you all the way to the beginning of a machine learning text -- it might need some linear optimization too?

Let me also mention two other resources:

• Victor Prasolov, Problems and solutions in linear algebra is a goldmine of interesting questions with solutions (although the latter are given in a very terse, Russian style). Note that it starts our the old-school way, with determinants and symmetric polynomials, as it is geared towards algebraists; but it includes a lot of matrix inequalities and positivity questions.

• Jean Gallier, Jocelyn Quaintance, Algebra, Topology, Differential Calculus, and Optimization Theory For Computer Science and Engineering is a comprehensive set of notes including (but not limited to) linear algebra. It gets all the way to SVMs, so it may be more relevant to your situation than many other sources. It has no solved exercises per se, though.

Answer 3

I really like Linear Algebra and It's Applications by Lay. I'm in an applied linear algebra class at the moment and I find the book very concise and helpful. It also has solutions in the back for odd numbered exercises.