# Linear algebra: Matrix identity

I am trying to understand a derivation and there is a matrix manipulation which I do not understand. So, there is the following derivative:

$$\frac{d}{dx} (x^T\Sigma^{-1}x)$$

Here x is a D $\times$ 1 vector (a D-dimensional vector) and $\Sigma$ is an invertible diagonal matrix. So, the derivation proceeds to say that the above quantity is:

$$2 \Sigma^{-1}x$$

If x was a scalar, I can see how this follows by chain rule but I got confused as to how it applies with the matrix transpose going on there. I was wondering if someone can point me to what identities are being used.

• What definition are you using for the derivative of a scalar quantity with respect to a vector? – Semiclassical Sep 23 '14 at 14:56
• a more correct answer is $2x^T\Sigma^{-1}$, a row vector – rych Oct 21 '14 at 4:40

Think of it coordinate-by-coordinate, then.

$$f(x_1,\ldots, x_n) = x^T \Sigma^{-1} x = \sum_{i,j} x_i (\Sigma^{-1})_{ij} x_j$$

For a particular component, say $x_k$, you have terms where $i=k$ and terms where $j=k$.

$$\dfrac{\partial f}{\partial x_k} = \sum_j (\Sigma^{-1})_{kj} x_j + \sum_i x_i (\Sigma^{-1})_{ik} = \sum_j \left((\Sigma^{-1})_{kj} + (\Sigma^{-1})_{jk})\right) x_j$$ and now use the fact that $\Sigma^{-1}$ is symmetric $$= 2 \sum_{j} (\Sigma^{-1})_{kj} x_j = 2 \left( \Sigma^{-1} x \right)_k$$

A mostly used identity $\frac{1}{\begin{pmatrix}v_1\\v_2\\..\\v_n\end{pmatrix}}=\frac{1}{n} \begin{pmatrix}v_1^{-1}\\v_2^{-1}\\..\\v_n^{-1}\end{pmatrix}$

With this $\left(\frac{1}{\vec v}\right)^\top\cdot \vec v = 1$

$$\frac{d}{d\vec x} \left(\vec x^\top\Sigma^{-1}\vec x\right) =\frac{1}{n} (\partial_{x_1}, \partial_{x_2}, \partial_{x_3}, ..., \partial_{x_n})^\top \left(\vec x^\top\Sigma^{-1}\vec x\right)$$ $$\Sigma^{-1}\vec x +\frac{d}{dx}\vec x^\top\Sigma^{-1}\vec x$$

Write the term $x^T\Sigma^{−1}x$ explicity as $\sum_i\sum_i x_ix_j\sigma_{ii}^*$ here $\Sigma^{-1}=(\sigma_{ij}^*)$. Then you have to calculate the gradient of this expression with respect to the vector $x$. Then you see it.

Let $A=\Sigma^{-1}$. In order to find the derivative let us calculate the h-linear term, $$$$\begin{split} (x+h)^TA(x+h)-x^TAx&=(x^T+h^T)A(x+h)-x^TAx\approx\\ &h^TAx+x^TAh= (h^TAx)^T+x^TAh=\\&x^T(A+A^T)h \end{split}$$$$ Hence the derivative is: $x^T(A+A^T)$, or $2x^TA$ for a symmetric $A$. Note that it is not a column- but a row-vector (or covector, or dual vector).