Let $\mathbf{\Sigma}$ be a symmetric positive definite matrix and let $\mathbf{L}$ be its lower Cholesky factor. Furthermore, let $\mathbf{D}$ be a diagonal matrix.

What is $$ \begin{align} \frac{\partial \text{vech} (\mathbf{L} \mathbf{D} \mathbf{L}^\top)}{\partial \text{vech}(\mathbf{\Sigma})^\top} \quad ? \end{align} $$

Probably this and this question is relevant.


1 Answer 1


We can use the chain rule,

$$ \begin{align} \frac{\partial \text{vech} (\mathbf{L} \mathbf{D} \mathbf{L}^\top)}{\partial \text{vech}(\mathbf{\Sigma})^\top} &= \frac{\partial \text{vech} (\mathbf{L} \mathbf{D} \mathbf{L}^\top)}{\partial \text{vech}(\mathbf{L})^\top} \frac{\partial \text{vech} (\mathbf{L})}{\partial \text{vech}(\mathbf{\Sigma})^\top} \\ &= \mathbf{G}^{+} \left( \mathbf{L} \mathbf{D} \otimes \mathbf{I} \right) \mathbf{F}^{\top} \left( \mathbf{G}^{+} \left(\mathbf{L} \otimes \mathbf{I}\right) \mathbf{F}^{\top} \right)^{-1}. \end{align} $$

To see this, let $\mathbf{G}$ be the duplication matrix, $\mathbf{G}^{+}$ its Moore-Penrose inverse, $\mathbf{F}$ the canonical elimination matrix (which consists of 0's and 1's only) and $\mathbf{K}$ the commutation matrix. Then $$ \begin{align} \mathrm{d} \mathrm{vech}\left( \mathbf{L} \mathbf{D} \mathbf{L}^{\top} \right) &= \mathbf{G}^{+} \, \, \mathrm{d} \mathrm{vec} \left( \mathbf{L} \mathbf{D} \mathbf{L}^{\top} \right) \\ &= \mathbf{G}^{+} \left( \mathrm{vec}\left( \mathrm{d} \mathbf{L} \mathbf{D} \mathbf{L}^{\top} \right) + \mathrm{vec}\left( \mathbf{L} \mathbf{D} \mathrm{d} \mathbf{L}^{\top} \right) \right) \\ &= \mathbf{G}^{+}\left(\mathbf{I} + \mathbf{K}_{pp}\right) \mathrm{vec}\left( \mathrm{d} \mathbf{L} \mathbf{D} \mathbf{L}^{\top} \right) \\ &= \mathbf{G}^{+} \left(\mathbf{I} + \mathbf{K}_{pp}\right) \left( \mathbf{L} \mathbf{D} \otimes \mathbf{I} \right) \mathrm{vec}\left( \mathrm{d} \mathbf{L} \right) \\ &= 2 \mathbf{G}^{+} \mathbf{G} \mathbf{G}^{+} \left( \mathbf{L} \mathbf{D} \otimes \mathbf{I} \right) \mathbf{F}^{\top} \mathrm{d} \mathrm{vech}\left( \mathbf{L} \right) \\ &= 2 \mathbf{G}^{+} \left( \mathbf{L} \mathbf{D} \otimes \mathbf{I} \right) \mathbf{F}^{\top} \mathrm{d} \mathrm{vech}\left( \mathbf{L} \right), \end{align} $$ and $$ \begin{align} \mathrm{d} \mathrm{vech}\left( \mathbf{\Sigma} \right) &= \mathbf{G}^{+} \mathrm{d} \mathrm{vec}\left( \mathbf{L} \mathbf{L}^{\top} \right) \\ &= \mathbf{G}^{+} \mathrm{vec}\left( \mathrm{d} \mathbf{L} \mathbf{L}^{\top} \right) + \mathbf{G}^{+} \mathrm{vec}\left( \mathbf{L} \mathrm{d} \mathbf{L}^{\top} \right) \\ &= \mathbf{G}^{+} \left(\mathbf{I} + \mathbf{K}_{pp}\right) \mathrm{vec}\left( \mathrm{d} \mathbf{L} \mathbf{L}^{\top} \right) \\ &= \mathbf{G}^{+} \mathbf{G} \mathbf{G}^{+} \mathrm{vec}\left( \mathrm{d} \mathbf{L} \mathbf{L}^{\top} \right) \\ &= 2 \mathbf{G}^{+} \left(\mathbf{L} \otimes \mathbf{I}\right) \mathrm{vec}\left( \mathrm{d} \mathbf{L} \right) \\ &= 2 \mathbf{G}^{+} \left(\mathbf{L} \otimes \mathbf{I}\right) \mathbf{F}^{\top} \mathrm{d} \mathrm{vech}\left( \mathbf{L} \right). \end{align} $$

  • $\begingroup$ $+\tt1\;$ You can eliminate one of the matrix variables by replacing $\,F\,$ with $\,G^+\;$ $\endgroup$
    – greg
    Jan 21, 2022 at 23:49
  • 1
    $\begingroup$ @greg, actually you have to keep in mind that $\mathbf{L}$ is a lower triangular matrix, which means that $\mathrm{vec} (\mathbf{L}) = \mathbf{F}^\top \mathrm{vech} (\mathbf{L})$. So we cannot replace $\mathbf{F}^\top$. $\endgroup$
    – stollenm
    Jan 22, 2022 at 11:38
  • $\begingroup$ Agreed. But $F^T$ isn't technically a duplication or elimination matrix. Instead it's a way to create a triangular matrix from a vector of the appropriate length. I think this answer (which you've basically re-derived) uses a really good notation for this distinction. $\endgroup$
    – greg
    Jan 22, 2022 at 17:44
  • $\begingroup$ @greg True, $\mathbf{F}^\top$ just maps the $\mathrm{vech}$ of a lower triangular into its $\mathrm{vec}$. The answer you are referring to is actually a re-derivation itself of a result by Lütkepohl (1989), Lemma 1. $\endgroup$
    – stollenm
    Jan 22, 2022 at 19:51
  • $\begingroup$ @stollenm Do you really need this computation ? I imagine you are computing analytically some gradient of a scalar-valued function $\phi$ involving the matrix $\mathbf{L} \mathbf{D} \mathbf{L}^T$ and you are interested in the gradient $\frac{\partial \phi}{\partial \mathbf{\Sigma}}$ at some point. There are other tools than vech to do this. If I am correct ,can you give us the whole picture ? $\endgroup$
    – Steph
    Jan 27, 2022 at 8:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .