5
$\begingroup$

Setup:

Let $k\in{}\mathbb{N},$ and let $\mathrm{M}_{k,k}(\mathbb{}{R})$ denote the set of $k\times{}k$ matrices over the field of real numbers.

Let $X\in{}\mathrm{M}_{k,k}(\mathbb{}{R})$ be a symmetric, positive definite matrix.

Then $X$ has $k$ positive eigenvalues $\lambda_{1},\dots{},\lambda_{k}$ with corresponding eigenvectors $v_{1},\dots{},v_{k}$.

The eigendecomposition/spectral decomposition of $X$ is:

$$X = V\Lambda{}V^{-1} = V\Lambda{}V',$$

where $\Lambda{}=\mathrm{diag}(\lambda_{1},\dots{},\lambda_{k})\in{}\mathrm{M}_{k,k}(\mathbb{}{R})$ is the diagonal matrix with the $k$ eigenvalues on the main diagonal and $V=(v_{1},\dots{},v_{k})\in{}\mathrm{M}_{k,k}(\mathbb{}{R})$ is the matrix whose $k$ columns are the orthonormal eigenvectors.

We define the natural matrix logarithm of $X$, denoted $\log{}(X)$, to be $$\log{}(X)=V\log{}(\Lambda{})V',$$ where $\log{}(\Lambda{})=\mathrm{diag}(\log{}(\lambda_{1}),\dots{},\log{}(\lambda_{k}))\in{}\mathrm{M}_{k,k}(\mathbb{}{R})$.

Question:

What, if it can be found, is the analytical form of the $[k(k+1)/2] \times{} [k(k+1)/2]$ Jacobian matrix

$$\frac{\partial{}\mathrm{vec}(X)}{\partial{}\mathrm{vec}(\log{}(X))'}$$

where $\mathrm{vec}(\cdot{})$ is the half-vectorization operator that stacks the lower triangular part of its square argument matrix.

(Background: This is a recurring problem in multivariate statistics when one adopts a "log-parameterization" of a covariance or precision matrix, which are both, by definition, symmetric and positive (semi-)definite.)

$\endgroup$
  • $\begingroup$ I have replaced in you very well presented question all "vech"s by "vec"s. $\endgroup$ – Jean Marie Feb 10 '17 at 19:57
  • $\begingroup$ @JeanMarie: It is quite common that $\mathrm{vech}(\cdot{})$ refers to the half-vectorization operator, whereas $\mathrm{vec}(\cdot{})$ refers to the full vectorization operator. As both $X$ and $\log{}(X)$ are symmetric, I am interested in $\partial{}\mathrm{vech}(X)/\partial{}\mathrm{vech}(\log{}(X))$. $\endgroup$ – MiMa Feb 11 '17 at 12:49
  • $\begingroup$ Maybe to see this better, note that $\partial{}\mathrm{vech}(X)/\partial{}\mathrm{vech}(\log{}(X))'$, is a $[k(k+1)/2]\times{}[k(k+1)/2]$ matrix, whereas $\partial{}\mathrm{vec}(X)/\partial{}\mathrm{vec}(\log{}(X))'$ would be a $k^{2}\times{}k^{2}$ matrix (But I am quite new to this forum and not fully familiar with the editing/changing questions system yet, so apologies if there are mistakes) $\endgroup$ – MiMa Feb 11 '17 at 12:57
5
$\begingroup$

Let $$\eqalign{ G &= \log(X) \implies X = e^G \cr }$$ and find the differential of $X$ via the power series of the exponential $$\eqalign{ dX=de^G &= d\,\Bigg[\sum_{i=0}^\infty \frac{G^i}{i!}\Bigg] \cr &= \sum_{i=1}^\infty \frac{1}{i!}\,\sum_{j=0}^{i-1}\,G^{j}\,dG\,G^{i-j-1}\cr }$$ Now apply vectorization $$\eqalign{ dx &=\Bigg[\sum_{i=1}^\infty \frac{1}{i!}\,\sum_{j=0}^{i-1}G^{i-j-1}\otimes G^{j}\Bigg]\,dg \cr \frac{\partial x}{\partial g} &= \sum_{i=1}^\infty \frac{1}{i!}\,\sum_{j=0}^{i-1}\,\Big[G^{i-j-1}\otimes G^{j}\,\Big] \cr\cr }$$ To change between vectorization and half-vectorization, multipy by Duplication and Elimination matrices of the appropriate dimensions $$\eqalign{ L_k\,dx &= L_k\Bigg[\frac{\partial x}{\partial g}\Bigg]D_kL_k\,dg \cr dx^\prime &= L_k\Bigg[\frac{\partial x}{\partial g}\Bigg]D_k\,dg^\prime \cr \frac{\partial x^\prime}{\partial g^\prime} &= L_k\Bigg[\frac{\partial x}{\partial g}\Bigg]D_k \cr \cr\cr }$$ P.S.

The reason I used $G$ for the log, was because I wanted to use $L$ for the elimination matrix.

In the vectorization of the power series I used the fact that $G$ is symmetric to omit some transpose operations.

For a specific matrix, you may have a finite polynomial available $$e^G = \sum_{i=0}^{k-1} c_iG^i$$which would simplify the main result to $$\frac{\partial x}{\partial g} = \sum_{i=1}^{k-1} c_i\,\sum_{j=0}^{i-1}\,\Big[G^{i-j-1}\otimes G^{j}\,\Big]$$

$\endgroup$
  • $\begingroup$ You are an expert in matrix calculations and such, and I don't doubt in many other things as well. I've never encountered half the methods you use to solve problems,thanks for the links on duplication and elimination. $\endgroup$ – Jeff Faraci Feb 15 '17 at 4:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.