Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm taking a machine learning course and it involves a lot of matrix computation like compute the derivatives of a matrix with respect to a vector term. In my linear algebra course these material is not covered and I browsed some book in the school library but didn't find something relevant to my problem. So can you recommend me a book which covers these matrix computations?


share|cite|improve this question
Related:… – Belgi Nov 2 '12 at 0:27
up vote 2 down vote accepted

I find the Matrix Cookbook handy. It's not a textbook, rather, to quote the introduction:

These pages are a collection of facts (identities, approximations, inequalities, relations, ...) about matrices and matters relating to them. It is collected in this form for the convenience of anyone who wants a quick desktop reference

share|cite|improve this answer
This seems good because it has a whole chapter on derivatives. "This section is covering differentiation of a number of expressions with respect to a matrix X." – littleO Nov 6 '12 at 10:45

I recommend two books

  1. Matrix Algebra from a statistician's perspective by David Harville. (chapter 15 covers derivatives of matrices with respect to matrices.)
  2. Matrix Differential Calculus with Applications in Statistics and Econometrics by Jan Magnus and Heinz Neudecker. (a huge portion of the book is about that particular question of differentiation.)

These two books are also excellent because they come with two additional monographs with a lot of solved problems (check for instance Matrix Algebra (Econometric Exercises, Vol. 1) by Karim Abadir and Jan Magnus which has a lot of solved problems on matrix differentiation). Also they have the additional advantage that they are directly relevant to machine learning.

share|cite|improve this answer

I suggest Matrix computation by Golub and van Loan.

share|cite|improve this answer
This needs about +10 – Emily Nov 2 '12 at 0:37

Bishop's excellent machine learning textbook, Pattern Recognition and Machine Learning, has an appendix on "Properties of Matrices". One section in this appendix (p. 697) is about Matrix Derivatives, and discusses formulas like this:

\begin{equation} \frac{\partial}{\partial A} \text{Tr}(AB) = B^T \end{equation}


\begin{equation} \frac{\partial}{\partial A} \ln | A | = \left(A^{-1} \right)^T, \end{equation} for example.

Peter Lax, one of the greatest living mathematicians, has an excellent book called Linear Algebra and its Applications. Chapter 9 is entitled "Calculus of Vector- and Matrix-Valued Functions".

share|cite|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.