1
$\begingroup$

Pardon me for not knowing LateX representation, I have following function, where $\mu$ and $\Sigma$ are both Matrices. $$ h = \mu^T \Sigma \mu $$ which is a function of $\alpha$, such that its derivative can be written as $$ \def\p#1#2{\frac{\partial #1}{\partial #2}} \p h\alpha = \left(\p \mu\alpha\right)^T\Sigma^{-1}\mu - \mu^T\Sigma^{-1}\p\Sigma\alpha \Sigma^{-1}\mu + \mu^T \Sigma \p \mu\alpha $$ I need to find out the Second derivative wrt $\alpha$. Please help, I searched through net, but couldn't fond any such property.

$\endgroup$
  • $\begingroup$ I suppose you have a typo in your function, the derivative is the one of $h = \mu^T \Sigma^{-1}\mu$. $\endgroup$ – martini May 29 '13 at 9:56
0
$\begingroup$

Assuming that $h = \mu^T \Sigma^{-1}\mu$, as suggested by the derivative, we have: Taking another derivative we apply the product rule as we did for the first and use the chain rule for $\Sigma^{-1}$, in using $\def\pd#1#2{\frac{\partial #1}{\partial #2}}\def\pdd#1#2{\frac{\partial^2 #1}{\partial #2^2}}$ $$ \pd{\Sigma^{-1}}{\alpha} = -\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1} $$ So for the first term of $\pd h\alpha$ we have $$ \pd{}\alpha\left(\pd{\mu^T}\alpha \Sigma^{-1}\mu \right) = \pdd{\mu^T}\alpha\Sigma^{-1}\mu - \pd{\mu^T}\alpha \Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\mu + \pd{\mu^T}\alpha \Sigma^{-1}\pd\mu\alpha $$ For the second term \begin{align*} &\!\!\!\!\!\!\!\pd{}\alpha\left( \mu^T\Sigma^{-1}\pd\Sigma\alpha \Sigma^{-1}\mu\right)\\ &= \pd{\mu^T}\alpha \Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\mu - \mu^T\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\mu + \mu^T\Sigma^{-1}\pdd\Sigma\alpha\Sigma^{-1}\mu\\& \ \ {} - \mu^T\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\mu + \mu^T\Sigma^{-1}\pd\Sigma\alpha \Sigma^{-1}\pd \mu\alpha \end{align*} And for the third term $$ \pd{}\alpha\left(\mu^T \Sigma^{-1}\pd\mu\alpha \right) = \pd{\mu^T}\alpha\Sigma^{-1}\pd\mu\alpha - {\mu^T}\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\pd\mu\alpha + {\mu^T} \Sigma^{-1}\pdd\mu\alpha $$ Adding everything, we should have \begin{align*} \pdd h \alpha &= \pdd{\mu^T}\alpha \Sigma^{-1}\mu - \mu^T\Sigma^{-1}\pdd\Sigma\alpha\Sigma^{-1}\mu + \mu^T\Sigma^{-1}\pdd\mu\alpha\\ &{}+ 2\left(\pd{\mu^T}\alpha \Sigma^{-1}\pd\mu\alpha - \pd{\mu^T}\alpha\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\mu-\mu^T\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\pd\mu\alpha + \mu^T\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\pd\Sigma\alpha\Sigma^{-1}\mu\right) \end{align*}

$\endgroup$
  • $\begingroup$ Thanks :) it was of great help, as we don't have it as our course syllabus, so just had to search net for everything. Could you suggest any tutorial, which contains all formulas of second derivatives, as a handbook or something? thanks in advance :) $\endgroup$ – shreelock May 29 '13 at 11:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.