Derivative by element from inverse matrices I have a matrix $\Sigma$ given by
\begin{equation*}
\Sigma = \begin{bmatrix}
\phi_{11} &\phi_{12}\\
\phi_{12} & \phi_{22}
\end{bmatrix}
\end{equation*}  
I'd like to calculate the partial derivative by element, for example, $\frac{\partial y^T\Sigma^{-1} y}{\partial \phi_{11}}$,where $y$ is a vector.
 A: First, computing the derivative of $\Sigma$ is straightforward
$$
\frac{\partial\Sigma}{\partial \phi_{11}} = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}=E_{11}
$$
To compute the derivative of $\Sigma^{-1}$ note that $\Sigma^{-1}\Sigma=I$, so by the product rule we have
$$
0 = \frac{\partial I}{\partial \phi_{ij}} = \frac{\partial \Sigma^{-1}\Sigma}{\partial \phi_{ij}}= \frac{\partial \Sigma^{-1}}{\partial \phi_{ij}}\Sigma + \Sigma^{-1}\frac{\partial \Sigma}{\partial \phi_{ij}} 
$$
Solving for $ \frac{\partial \Sigma^{-1}}{\partial \phi_{ij}}$ in this equation we get the answer that
$$
\frac{\partial \Sigma^{-1}}{\partial \phi_{ij}} = -\Sigma^{-1}\frac{\partial \Sigma}{\partial \phi_{ij}}\Sigma^{-1}=-\Sigma^{-1}E_{ij}\Sigma^{-1}
$$
If $y$ is independent of $\phi_{ij}$ that means the derivative of your expression would be
$$
-y^{\top}\Sigma^{-1}E_{ij}\Sigma^{-1}y
$$
Otherwise you would have to use the product rule.
A: You can compute $\Sigma^{-1}$ as $$\Sigma^{-1}=\frac{1}{\phi_{11}\phi_{22}-\phi_{21}\phi_{12}}\begin{pmatrix}\phi_{22}&-\phi_{12}\\-\phi_{21}&\phi_{11}\end{pmatrix}$$
Then you can multiply this by $y$ and $y^T$ as you described, giving you a scalar. 
$$\frac{1}{\phi_{11}\phi_{22}-\phi_{21}\phi_{12}}\left(\phi_{22}y_1^2-\phi_{12}y_1y_2+\phi_{11}y_2^2-\phi_{21}y_1y_2\right)$$
Then differentiate each term.
