$\renewcommand{\v}[1]{\mathrm{vec}\left(#1\right)} \renewcommand{\m}[1]{\mathbf{#1}} \renewcommand{\trace}[1]{\mathrm{trace}\left(#1\right)} \renewcommand{\diag}[1]{\mathrm{diag}\left(#1\right)}$

Suppose we have the diagonal matrix $\mathbf{D} = Diag(\mathbf{1}^{T}\mathbf{H})$, $1$ is a column vector with ones.

How can we calculate the partial derivative of the following wrt matrix calculus? ($A$ is known matrix.) $$\frac{\partial ({ \mathbf D^{-1} \mathbf A)}}{\partial\m H}$$

Since now, I have reached the following using matrix cookbook:

$$\frac{\partial ({ \mathbf D^{-1} \mathbf A)}}{\partial\m H} = -\mathbf{A} \mathbf{D}^{-1} \frac{\partial { \mathbf D}}{\partial\m H} \mathbf{D}^{-1} = -\mathbf{A} \mathbf{D}^{-1} \mathbf{1}^{T} \mathbf{J} \mathbf{D}^{-1} $$

I think I am missing something with 1 and J.

  • $\begingroup$ What is $H$ in the line $Diag(1^TH)$? Or is it meant to be $A$? $\endgroup$ – user1936752 Mar 20 at 13:11
  • $\begingroup$ H is the uknown matrix. That's why the partial derivative is wrt H. And D is basically a diagonal matrix with its diagonal entries be the sum of each column of H. (that's what $\mathbf{1}^{T}$ does). $\endgroup$ – OliveR Mar 20 at 19:51

Specify the dimensions of all the vectors and matrices involved. $$\eqalign{ A & &\in {\mathbb R}^{n\times p} \cr H & &\in {\mathbb R}^{m\times n} \cr h &= {\rm vec}(H) &\in {\mathbb R}^{mn\times 1} \cr v &= H^T1_m &\in {\mathbb R}^{n\times 1} \cr L &= (I_n\otimes 1_n)\odot(1_n\otimes I_n) &\in {\mathbb R}^{n^2\times n} \cr B &= {\rm Diag}(v) &\in {\mathbb R}^{n\times n} \cr b &= {\rm vec}(B) = Lv &\in {\mathbb R}^{n^2\times 1} \cr &= LH^T1_m \cr &= {\rm vec}(1_m^THL^T) \cr &= (L\otimes 1_m^T)\,h \cr }$$ where $I_n$ is the $n\times n$ identity matrix, $1_n$ is the all-ones vector of length $n$, $\odot$ is the Hadamard product, and $\otimes$ is the Kronecker product.

The function of interest is matrix-valued, so it must be flattened/vectorized in order to express the resulting derivative as a matrix (instead of a fourth-order tensor).

The steps are to calculate the function differential, vectorize it, and formulate the matrix derivative. $$\eqalign{ F &= B^{-1}A\cr dF &= -B^{-1}\,dB\,B^{-1}A \cr &= -B^{-1}\,dB\,F \cr {\rm vec}(dF) &= -(F^T\otimes B^{-1})\,{\rm vec}(dB) \cr df &= -(F^T\otimes B^{-1})\,db \cr &= -(F^T\otimes B^{-1})\,(L\otimes 1_m^T)\,dh \cr \frac{\partial f}{\partial h} &= -(F^T\otimes B^{-1})\,(L\otimes 1_m^T) &= \frac{\partial \,{\rm vec}(F)}{\partial \,{\rm vec}(H)} \cr }$$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.