# Derivative of diagonal matrix expression: $f(X)=\text{diag}(X)M^T\text{diag}^{-1}(MX)$

Let X be a vector $$\mathbb{R}^{n\times1}$$ and M be a constant matrix $$\mathbb{R}^{n\times n}$$, and given the function $$f(X)$$ how could i find the derivative of $$f(X)$$ with respect to $$X$$? In the expression diag($$X$$) represents the diagonal matrix of $$X$$.

$$f(X)=\text{diag}(X)M^T\text{diag}^{-1}(MX)$$

• Can you check the parenthesis is correct? The shape of output from $f(X)$ is $n\times n$? May 14 at 3:32

For typing convenience, define the variables \eqalign{ \def\bx{\boxtimes} \def\LR#1{\left(#1\right)} \def\qiq{\quad\implies\quad} \def\A{A^{-1}} \def\o{{\tt1}} \def\p{\partial} \def\grad#1#2{\frac{\p #1}{\p #2}} \def\bbR#1{{\mathbb R}^{#1}} \def\m#1{\left[\begin{array}{r}#1\end{array}\right]} \def\D{\operatorname{Diag}} \def\v{\operatorname{vec}} X &= {\rm Diag}(x) \\ A &= {\rm Diag}(Mx) = A^T \\ F &= XM^T\A \\ } and the symbol $$\bx$$ for the columnar Khatri-Rao product \eqalign{ A &= \m{a_1&a_2&\ldots&a_n} \;&\in\bbR{m\times n} \\ B^T &= \m{b_1&\,b_2&\ldots&\,b_n} \;&\in\bbR{p\times n} \\ C &= \m{c_1&\,c_2&\ldots&\,c_n} \;&\in\bbR{mp\times n} \\ C &= \LR{B^T\bx A} \;\iff\; c_k &= \LR{b_k\otimes a_k} \\ } where $$\otimes$$ is the Kronecker product. The one remarkable property of this product is its ability to vectorize products involving a diagonal matrix \eqalign{ &\v\!\big(A\,\D(x)\;B\,\big) \;=\; (B^T\bx A)\,x \\ } Use the above notation to calculate the gradient. \eqalign{ dF &= dX\,M^T\A + XM^T\,d\A \\ &= I_n\,\D(dx)\;M^T\A - XM^T\A\,\D(M\,dx)\;\A \\ \\ df &= \v(dF) \\ &= \LR{\A M\bx I_n}dx - \LR{\A\bx XM^T\A}M\,dx \\ \\ \grad{f}{x} &= \LR{\A M\bx I_n} - \LR{\A\bx F}M \\ \\ }

• Thank you for your answer, I would like to ask you what is In. Is it the Identity Matrix? Also i would like to ask you about the dimension of the derivative, is it $n^2$ x n? And i would like to request you for recommendatios abouts books to study further this topic. Thank you in advance May 14 at 16:45
• The standard text is probably Magnus and Neudecker's Matrix Differential Calculus And yes, $I_n$ is the identity matrix. $\,(A,F,I_n,X)$ are all $n\times n$ matrices. The vector $f$ has length $n^2$ while the vector $x$ has a length of $n.\;$
– greg
May 14 at 16:49

One way to simplify the matter is to use subscripts (einstein summation rule) when dealing with matrix function and derivative.

For example your function could be re-writen as such

$$f(X)_{ij}=X_i M_{ji} (\sum_l M_{jl}X_l)^{-1}$$

Then you could figure out this partial derivative tensor based on that $$\partial f(X)_{ij}/\partial X_k$$