# Is there a rigorous definition for matrix derivatives?

I know that,

A function $$f: \mathbb{R}^n \to \mathbb{R}$$ is said to be differentiable at $$x$$ if there exists a vector $$v$$ such that, $$\lim_{h \to 0} \frac{f(x+h) - f(x) - v^Th} {\|h\|} = 0.$$ When $$v$$ exists, it is given by the "gradient" $$\nabla f(x) = \left(\frac{\partial f}{\partial x_1}, ..., \frac{\partial f}{\partial x_n}\right)(x)$$

Does there exist a similar definition for "matrix derivative"

https://en.wikipedia.org/wiki/Matrix_calculus#Derivatives_with_matrices

• As far as physics is concerned, it is defined in the same way as in systems of ODE's. For example, the equation for an open system follows a master equation of the form $\dot{\rho}=\mathcal{L}(\rho)$, where $\rho$ is a matrix (a density matrix) and $\mathcal{L}$ is the "Lindbladian" which is a superoperator that takes operators into operators (in finite Hilbert spaces, $\rho$ can be represented as a matrix). In order to solve this equation, say numerically, one can consider that every $\rho_{ij}$ fulfills an ordinary ODE. Overall, one ends up with a system of ODEs to be solved. – user2820579 Jan 21 at 21:28
• Replace $v^T$ with a linear map $L\colon \Bbb R^n \to \Bbb R^k$. – Ivo Terek Jan 21 at 21:54
• Just reiterating what has been said here, but what you're looking for is the Frechet derivative. See my answer here which later also provides a sample calculation. Also, you may want to take a look at this for the motivation of the general definition. – peek-a-boo Jan 22 at 8:14

Matrix derivation is just a particular case of Fréchet derivative between two Banach spaces. Which by the way is very similar in term of definition to the definition of the derivative of a function $$f : \mathbb R^n \to \mathbb R$$ provided in the question.
Applied to matrix derivatives, you just have to consider a map $$f : V \to M$$ where $$M$$ is a linear space of matrices endowed with the norm of your choice and $$V$$ a Banach space that can be (or not) of finite dimension.
Yes, you can use a very similar definition. First of all, the map $$h \mapsto v^\top h$$ encodes an arbitrary linear map on $$\mathbb R^n$$. For matrices, you can substitute it by the Frobenius inner product, e.g., $$A \mapsto (A,B)_F := \sum_{i,j = 1}^n A_{ij} B_{ij},$$ where $$B \in \mathbb R^{n \times n}$$ is fixed.
Thus, for $$f \colon \mathbb R^{n \times n} \to \mathbb R$$ you can define $$\nabla f(A)$$ to be the (unique) matrix $$B \in \mathbb R^{n \times n}$$ (if it exists), that satisfies $$\lim_{H \to 0} \frac{f(A + H) - f(A) - (B,H)_F}{\|H\|} = 0.$$