2
$\begingroup$

Let X be a vector $\mathbb{R}^{n\times1}$ and M be a constant matrix $\mathbb{R}^{n\times n}$, and given the function $f(X)$ how could i find the derivative of $f(X)$ with respect to $X$? In the expression diag($X$) represents the diagonal matrix of $X$.

$$f(X)=\text{diag}(X)M^T\text{diag}^{-1}(MX)$$

Thank you in advance.

$\endgroup$
1
  • $\begingroup$ Can you check the parenthesis is correct? The shape of output from $f(X)$ is $n\times n$? $\endgroup$ May 14 at 3:32

2 Answers 2

2
$\begingroup$

For typing convenience, define the variables $$\eqalign{ \def\bx{\boxtimes} \def\LR#1{\left(#1\right)} \def\qiq{\quad\implies\quad} \def\A{A^{-1}} \def\o{{\tt1}} \def\p{\partial} \def\grad#1#2{\frac{\p #1}{\p #2}} \def\bbR#1{{\mathbb R}^{#1}} \def\m#1{\left[\begin{array}{r}#1\end{array}\right]} \def\D{\operatorname{Diag}} \def\v{\operatorname{vec}} X &= {\rm Diag}(x) \\ A &= {\rm Diag}(Mx) = A^T \\ F &= XM^T\A \\ }$$ and the symbol $\bx$ for the columnar Khatri-Rao product $$\eqalign{ A &= \m{a_1&a_2&\ldots&a_n} \;&\in\bbR{m\times n} \\ B^T &= \m{b_1&\,b_2&\ldots&\,b_n} \;&\in\bbR{p\times n} \\ C &= \m{c_1&\,c_2&\ldots&\,c_n} \;&\in\bbR{mp\times n} \\ C &= \LR{B^T\bx A} \;\iff\; c_k &= \LR{b_k\otimes a_k} \\ }$$ where $\otimes$ is the Kronecker product. The one remarkable property of this product is its ability to vectorize products involving a diagonal matrix $$\eqalign{ &\v\!\big(A\,\D(x)\;B\,\big) \;=\; (B^T\bx A)\,x \\ }$$ Use the above notation to calculate the gradient. $$\eqalign{ dF &= dX\,M^T\A + XM^T\,d\A \\ &= I_n\,\D(dx)\;M^T\A - XM^T\A\,\D(M\,dx)\;\A \\ \\ df &= \v(dF) \\ &= \LR{\A M\bx I_n}dx - \LR{\A\bx XM^T\A}M\,dx \\ \\ \grad{f}{x} &= \LR{\A M\bx I_n} - \LR{\A\bx F}M \\ \\ }$$

$\endgroup$
2
  • $\begingroup$ Thank you for your answer, I would like to ask you what is In. Is it the Identity Matrix? Also i would like to ask you about the dimension of the derivative, is it $n^2$ x n? And i would like to request you for recommendatios abouts books to study further this topic. Thank you in advance $\endgroup$ May 14 at 16:45
  • $\begingroup$ The standard text is probably Magnus and Neudecker's Matrix Differential Calculus And yes, $I_n$ is the identity matrix. $\,(A,F,I_n,X)$ are all $n\times n$ matrices. The vector $f$ has length $n^2$ while the vector $x$ has a length of $n.\;$ $\endgroup$
    – greg
    May 14 at 16:49
1
$\begingroup$

One way to simplify the matter is to use subscripts (einstein summation rule) when dealing with matrix function and derivative.

For example your function could be re-writen as such

$$ f(X)_{ij}=X_i M_{ji} (\sum_l M_{jl}X_l)^{-1} $$

Then you could figure out this partial derivative tensor based on that $$ \partial f(X)_{ij}/\partial X_k $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.