Can somebody tell me how to arrive at the following expressions

\begin{align} \mathbf{D} &= (\mathbf{C}^{-1}\mathbf{t}\mathbf{t}^{T}\mathbf{C}^{-1}-\mathbf{C}^{-1}) \mathbf{\Phi} \mathbf{A}^{-1} \\ &= \beta[(\mathbf{t}-\mathbf{y})\boldsymbol{\mu}^{T} - \mathbf{\Phi}\Sigma] \end{align}

where \begin{equation} \mathbf{C} = \beta^{-1}\mathbf{I} + \mathbf{\Phi} \mathbf{A}^{-1} \mathbf{\Phi}^{T}, \\ \boldsymbol{\mu} = \beta^{-1}\mathbf{\Sigma}\mathbf{\Phi}^{T}\mathbf{t}, \\ \boldsymbol{\Sigma} = \left( \mathbf{A} + \beta \mathbf{\Phi}^{T} \mathbf{\Phi} \right)^{-1} \end{equation}

which contain the derivatives of the following function \begin{equation} \mathcal{L} = -\frac{1}{2}\big[N\log 2\pi + \log\left|\mathbf{C}\right| + \mathbf{t}^{T}\mathbf{C}^{-1}\mathbf{t} \big] \end{equation}

with respect to the elements $\phi_{mn}$ of the matrix $\mathbf{\Phi}$ (i.e. $\mathbf{D}_{mn} = \partial \mathcal{L} / \partial \phi_{mn}$)?

Note that bold small letters e.g $\mathbf{t}$ denote $N\times 1$ vectors, bold capital letters $N\times N$ matrices, and $\mathbf{A}$ is diagonal.

Also, the author of the paper where I found it makes frequent use of the following identities (by the Woodbury inversion lemma) which might be helpful:

\begin{align} \mathbf{C}^{-1} = (\beta^{-1}\mathbf{I} + \mathbf{\Phi} \mathbf{A}^{-1} \mathbf{\Phi}^{T} )^{-1} = \beta \mathbf{I} - \beta \mathbf{\Phi}\left( \mathbf{A} + \beta \mathbf{\Phi}^{T} \mathbf{\Phi} \right)^{-1} \mathbf{\Phi}^{T} \beta = \beta \mathbf{I} - \beta \mathbf{\Phi} \mathbf{\Sigma} \mathbf{\Phi}^{T} \beta \end{align}


Before we begin, note that $\bf C$ is symmetric, since $\bf A$ is diagonal and therefore symmetric.

Also note that the term $\,\,{\bf {\rm log}({\rm det}({\bf C}))}\,\,$ can be replaced by $\,\,{\bf {\rm tr}({\rm log}({\bf C}))}\,\,$ which is easier to work with. And the final term has been written in terms of the Frobenius product, for the same reason.

So start with the expression $$ -2\, L = N\,{\rm log}(2\pi) + {\rm tr}({\rm log}({\bf C})) + {\bf C}^{-1}:{\bf tt}^T $$ and take the differential of both sides $$\eqalign{ -2\,dL &= {\bf C}^{-1}:d{\bf C} - {\bf tt}^T:({\bf C}^{-1}\,d{\bf C}\,{\bf C}^{-1}) \cr &= {\bf C}^{-1}:d{\bf C} - {\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}:d{\bf C} \cr &= ({\bf C}^{-1} - {\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}):d{\bf C} \cr }$$ Now find an expression for $d{\bf C}$ in terms of $d {\bf \Phi}$. $$\eqalign{ {\bf C} &= \beta^{-1}{\bf I} + {\bf \Phi} {\bf A}^{-1}{\bf \Phi}^T \cr d{\bf C} &= d\Phi {\bf A}^{-1}{\bf \Phi}^T + {\bf \Phi} {\bf A}^{-1}d{\bf \Phi}^T \cr &= 2\,\,{\rm sym}(d{\bf \Phi} {\bf A}^{-1}{\bf \Phi}^T) \cr }$$ Substituting $$\eqalign{ -2\,dL &= ({\bf C}^{-1}-{\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}):2\,{\rm sym}(d{\bf \Phi} {\bf A}^{-1}{\bf \Phi}^T) \cr &= 2\,{\rm sym}({\bf C}^{-1} - {\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}):d{\bf \Phi} {\bf A}^{-1}{\bf \Phi}^T \cr &= 2\,({\bf C}^{-1} - {\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}):d{\bf \Phi} {\bf A}^{-1}{\bf \Phi}^T \cr &= 2\,({\bf C}^{-1} - {\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}){\bf \Phi} {\bf A}^{-1}:d{\bf \Phi} \cr\cr dL &= ({\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}-{\bf C}^{-1}){\bf \Phi} {\bf A}^{-1}:d{\bf \Phi} \cr\cr }$$ So the derivative is $$\eqalign{ {\bf D} &= \frac{\partial L}{\partial {\bf \Phi}} &= ({\bf C}^{-1}{\bf tt}^T{\bf C}^{-1}-{\bf C}^{-1}){\bf \Phi} {\bf A}^{-1} \cr }$$ which you can reformulate in terms of {$y,\mu,\Sigma$, etc} if you wish.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.