Linear algebra: Matrix identity I am trying to understand a derivation and there is a matrix manipulation which I do not understand. So, there is the following derivative:
$$
\frac{d}{dx} (x^T\Sigma^{-1}x)
$$
Here x is a D $\times$ 1 vector (a D-dimensional vector) and $\Sigma$ is an invertible diagonal matrix. So, the derivation proceeds to say that the above quantity is:
$$
2 \Sigma^{-1}x
$$
If x was a scalar, I can see how this follows by chain rule but I got confused as to how it applies with the matrix transpose going on there. I was wondering if someone can point me to what identities are being used. 
 A: Think of it coordinate-by-coordinate, then.
$$f(x_1,\ldots, x_n) = x^T \Sigma^{-1} x = \sum_{i,j} x_i (\Sigma^{-1})_{ij} x_j$$
For a particular component, say $x_k$, you have terms where $i=k$ and terms where $j=k$.
$$ \dfrac{\partial f}{\partial x_k} = \sum_j (\Sigma^{-1})_{kj} x_j + \sum_i x_i (\Sigma^{-1})_{ik} = \sum_j \left((\Sigma^{-1})_{kj} + (\Sigma^{-1})_{jk})\right) x_j $$
and now use the fact that $\Sigma^{-1}$ is symmetric
$$ = 2 \sum_{j} (\Sigma^{-1})_{kj} x_j = 2 \left( \Sigma^{-1} x \right)_k $$
A: A mostly used identity $\frac{1}{\begin{pmatrix}v_1\\v_2\\..\\v_n\end{pmatrix}}=\frac{1}{n} \begin{pmatrix}v_1^{-1}\\v_2^{-1}\\..\\v_n^{-1}\end{pmatrix}$
With this $\left(\frac{1}{\vec v}\right)^\top\cdot \vec v = 1$
$$\frac{d}{d\vec x} \left(\vec x^\top\Sigma^{-1}\vec x\right) =\frac{1}{n} (\partial_{x_1}, \partial_{x_2}, \partial_{x_3}, ..., \partial_{x_n})^\top \left(\vec x^\top\Sigma^{-1}\vec x\right)$$
$$ \Sigma^{-1}\vec x +\frac{d}{dx}\vec x^\top\Sigma^{-1}\vec x$$
A: Write the term $x^T\Sigma^{−1}x$ explicity as
$\sum_i\sum_i x_ix_j\sigma_{ii}^*$ here $\Sigma^{-1}=(\sigma_{ij}^*)$. Then
you have to calculate the gradient of this expression with respect to the vector $x$. Then you see it.
A: Let $A=\Sigma^{-1}$.
In order to find the derivative let us calculate the h-linear term,
$$\begin{equation}
\begin{split} 
(x+h)^TA(x+h)-x^TAx&=(x^T+h^T)A(x+h)-x^TAx\approx\\
&h^TAx+x^TAh= (h^TAx)^T+x^TAh=\\&x^T(A+A^T)h
\end{split}
\end{equation}$$
Hence the derivative is: $x^T(A+A^T)$, or $2x^TA$ for a symmetric $A$.
Note that it is not a column- but a row-vector (or covector, or dual vector).
