A symmetric matrix $S=S^T$ has orthonormal eigenvectors $\vec{v}_1$ to $\vec{v}_n$. Then any vector $\vec{x}$ can be written as a combination $\vec{x} = c_1 \vec{v}_1+ \cdots + c_n \vec{x}_n$. Explain this formula: $\vec{x}^{\,T}S\vec{x} = \lambda_1 c_1^2+ \cdots + \lambda_nc_n^2$

My explanation:

$\vec{x}$ can be written as $$ \begin{bmatrix} \vec{v}_1 & \cdots & \vec{v}_n \end{bmatrix}_{nxn} \begin{bmatrix} c_1 \\ \vdots \\ c_n \end{bmatrix}_{\,nx1} = V\vec{c} $$ Consider, $$ \vec{x}^{\,T}S\vec{x} = \vec{c}^{\,T}(V^{\,T}SV)\vec{c} = \vec{c}^{\,T}\Lambda\vec{c} = \lambda_1 c_1^2 + \cdots + \lambda c_n^2 $$ We see that orthonormal eigenvectors of $S$ multiply to make the diagonal eigenvlaue matrix. We then end up with the inner product of the vector weightings, the $c^2$ terms, scaled by their respective eigenvalues.

I feel pretty good about my explanation. Just the part that I don't fully understand is why $V^{\,T}SV=\Lambda$... It seems to me like it should equal something more along the lines of $\Lambda^T \Lambda$ since each eigenvector should have been scaled when right multiplied because $A\vec{v} = \lambda \vec{v}$.

Is it true that $V^{\,T}SV=\Lambda$? If it is, a little help seeing why would be appreciated.

A way I thought about it.

$$ V^TSV = V^T(SV) = V^T \begin{bmatrix} \lambda_1 \vec{v_1} & ... & \lambda_n \vec{v_n} \end{bmatrix} $$ The way to think about $SV$ is that each column of $V$ is an eigenvector that multiplies the columns of $S$. We already know what this combination will equal by the definition of an eigenvector $A\vec{v} = \lambda\vec{v}$. Finally, when right multiplying by $V^T$ remember that $\vec{v}_i^{\,T}\vec{v_j}=1$ is $1$ when $i=j$ and $0$ when $i\neq j$ by definition of being orthonormal vectors.

Clearly we end up with $\Lambda$ as the result.


1 Answer 1


Yes, it is true that $V^TSV = \Lambda$. There are several ways that we can understand why this holds; one way is to compare what each matrix "does" to a column-vector.

Let $\vec c$ denote the column vector $\vec c = (c_1,\dots,c_n)$. Verify that $\Lambda \vec c = (\lambda_1 c_1,\dots,\lambda_n c_n)$.

Now, we consider the product $V^TSV \vec c = V^T(S(V\vec c))$. We find that $$ V \vec c = \pmatrix{\vec v_1 & \cdots & \vec v_n} \pmatrix{c_1 \\ \vdots \\ c_n} = c_1 \vec v_1 + \cdots + c_n \vec v_n. $$ To put things another way, the role of the $V$ is to interpret the entries of $\vec c$ as the coefficients of the vectors $\vec v_1,\dots,\vec v_n$. From there, we see that because each $v_i$ is an eigenvector of $S$, we have $$ S(V\vec c) = S(c_1 \vec v_1 + \cdots + c_n \vec v_n) = c_1 \lambda_1 \vec v_1 + \cdots + c_n \lambda_n \vec v_n. $$ Finally, note that (because $V$ is orthogonal) $V^T$ is the inverse of $V$. So, just as $V$ interprets the "input vector" as a list of coefficients for the $v_i$, $V^T$ takes a vector and gives us the the list of coefficients for the $v_i$ as its output. That is, $$ V^T(S(V\vec c)) = (c_1 \lambda_1) \vec v_1 + \cdots + (c_n \lambda_n) \vec v_n = (c_1 \lambda_1,\dots,c_n \lambda_n). $$ So indeed, $V^TSV$ and $\Lambda$ describe the same transformation and are therefore the same matrix.

We can think of $V^TSV$ as an altered version of $S$ where, instead of thinking of the entries of a vector (both input and output) as the literal coordinates of a vector, we interpret them as the coefficients of our vectors $v_i$. In terms of terminology that you might already have heard before, we say that $\Lambda$ is the matrix of $S$ "after a change of basis".

  • $\begingroup$ Solid, lovely explanation. I was thinking about it while I was falling asleep yesterday... I'll edit my question to show how I came to the same conclusion as you $\endgroup$ Sep 14, 2020 at 16:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .