# Second Derivative of a matrix

Pardon me for not knowing LateX representation, I have following function, where $\mu$ and $\Sigma$ are both Matrices. $$h = \mu^T \Sigma \mu$$ which is a function of $\alpha$, such that its derivative can be written as $$\def\p#1#2{\frac{\partial #1}{\partial #2}} \p h\alpha = \left(\p \mu\alpha\right)^T\Sigma^{-1}\mu - \mu^T\Sigma^{-1}\p\Sigma\alpha \Sigma^{-1}\mu + \mu^T \Sigma \p \mu\alpha$$ I need to find out the Second derivative wrt $\alpha$. Please help, I searched through net, but couldn't fond any such property.

• I suppose you have a typo in your function, the derivative is the one of $h = \mu^T \Sigma^{-1}\mu$. – martini May 29 '13 at 9:56

Assuming that $h = \mu^T \Sigma^{-1}\mu$, as suggested by the derivative, we have: Taking another derivative we apply the product rule as we did for the first and use the chain rule for $\Sigma^{-1}$, in using $\def\pd#1#2{\frac{\partial #1}{\partial #2}}\def\pdd#1#2{\frac{\partial^2 #1}{\partial #2^2}}$ $$\pd{\Sigma^{-1}}{\alpha} = -\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}$$ So for the first term of $\pd h\alpha$ we have $$\pd{}\alpha\left(\pd{\mu^T}\alpha \Sigma^{-1}\mu \right) = \pdd{\mu^T}\alpha\Sigma^{-1}\mu - \pd{\mu^T}\alpha \Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\mu + \pd{\mu^T}\alpha \Sigma^{-1}\pd\mu\alpha$$ For the second term \begin{align*} &\!\!\!\!\!\!\!\pd{}\alpha\left( \mu^T\Sigma^{-1}\pd\Sigma\alpha \Sigma^{-1}\mu\right)\\ &= \pd{\mu^T}\alpha \Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\mu - \mu^T\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\mu + \mu^T\Sigma^{-1}\pdd\Sigma\alpha\Sigma^{-1}\mu\\& \ \ {} - \mu^T\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\mu + \mu^T\Sigma^{-1}\pd\Sigma\alpha \Sigma^{-1}\pd \mu\alpha \end{align*} And for the third term $$\pd{}\alpha\left(\mu^T \Sigma^{-1}\pd\mu\alpha \right) = \pd{\mu^T}\alpha\Sigma^{-1}\pd\mu\alpha - {\mu^T}\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\pd\mu\alpha + {\mu^T} \Sigma^{-1}\pdd\mu\alpha$$ Adding everything, we should have \begin{align*} \pdd h \alpha &= \pdd{\mu^T}\alpha \Sigma^{-1}\mu - \mu^T\Sigma^{-1}\pdd\Sigma\alpha\Sigma^{-1}\mu + \mu^T\Sigma^{-1}\pdd\mu\alpha\\ &{}+ 2\left(\pd{\mu^T}\alpha \Sigma^{-1}\pd\mu\alpha - \pd{\mu^T}\alpha\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\mu-\mu^T\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\pd\mu\alpha + \mu^T\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\mu\right) \end{align*}