# Derivative of matrix-valued function with respect to matrix input

I have the expression

$$\bf \phi = \bf X W$$

where $\bf X$ is a $20 \times 10$ matrix, $\bf W$ is a $10 \times 5$ matrix.

How can I calculate $\frac{d\phi}{d\bf W}$? What is the dimension of the result?

• Have you tried anything so far? What are your thoughts? Commented Aug 20, 2017 at 11:38
• I would say that the result could be a tensor, but I am not sure about that. Commented Aug 20, 2017 at 11:40
• This is related to a programmign task, I deleted the b vector since it's not important to the aim of the question. Commented Aug 20, 2017 at 11:44

Let function $\mathrm F : \mathbb R^{n \times p} \to \mathbb R^{m \times p}$ be defined as follows

$$\rm F (X) := A X$$

where $\mathrm A \in \mathbb R^{m \times n}$ is given. The $(i,j)$-th entry of the output is

$$f_{ij} (\mathrm X) = \mathrm e_i^\top \mathrm A \, \mathrm X \, \mathrm e_j = \mbox{tr} \left( \mathrm e_j \mathrm e_i^\top \mathrm A \, \mathrm X \right) = \langle \mathrm A^\top \mathrm e_i \mathrm e_j^\top, \mathrm X \rangle$$

Hence,

$$\partial_{\mathrm X} \, f_{ij} (\mathrm X) = \color{blue}{\mathrm A^\top \mathrm e_i \mathrm e_j^\top}$$

• Rodrigo I'd like to deepen this kind of topic, do you have any reference to some materials? Thanks. Commented Aug 23, 2017 at 8:23
• I only know the matrix calculus tag. Commented Aug 23, 2017 at 8:38

There is a similar question.

Also, you could define it

$$C = \frac{\partial \phi}{\partial W}$$

where C is a 4D matrix (or tensor) with

$$C_{a,b,c,d} = \frac{\partial \phi_{a,b}}{\partial W_{c,d}}$$

Actually, when derivatives are expressed as matrices, for example, $f=x^TAx$ where $x\in R^{n\times1}, A\in R^{n\times n}$, you could think of $\frac{\partial f}{\partial A}$ as

$$\left[\frac{\partial f}{\partial A}\right]_{ij} = \frac{\partial f}{\partial A_{ij}}$$