1
$\begingroup$

I am hung-up on one part of the proof of the spectral theorem for real matrices. The proof by induction makes perfect sense, but when I try to write it using only matrices I'm struggling to show that the eigenvalues of $B$ are the same as $A$.

Quick proof of spectral theorem of real, linear operators on $\mathbb{R}^n$ by induction


Here's a quick rundown of the proof by induction that I am referring to:

Base case. Let let $V$ be an inner product space of $\dim V = 1$ and $T$ a self-adjoint linear operator $T: V \to V$. Then $T$ has a real eigenvalue $\lambda$ and corresponding real eigenvector $\mathbf{v}\in \mathbb{R}$. Now, $\frac{\mathbf{v}}{\| \mathbf{v} \|}$ spans $\mathbb{R}$.

Inductive step. Now let $\dim V = n+1$. $T$ again has a real eigenvalue $\lambda$ and corresponding real eigenvector $\mathbf{v}$. Let $U = \text{span} \{ \mathbf{v} \}$, so $U^\perp = \{ \mathbf{u} \in V \; | \; \langle \mathbf{u}, \mathbf{v} \rangle = 0\}$.

Note that $\dim U^\perp = n$ and that $U^\perp$ is invariant under $T$ as $$ \langle T(\mathbf{u}), \mathbf{v} \rangle = \langle \mathbf{u}, T(\mathbf{v}) \rangle = \langle \mathbf{u}, \lambda \mathbf{v} \rangle = \lambda \langle \mathbf{u}, \mathbf{v} \rangle = 0 \; , \; \forall \mathbf{u} \in U^\perp $$ Define $E : U^\perp \to U^\perp$, where $E = \left. T \right|_{U^\perp}$, then $E$ is self-adjoint $$ \langle E(\mathbf{u}_1) , \mathbf{u}_2 \rangle = \langle T(\mathbf{u}_1) , \mathbf{u}_2 \rangle = \langle \mathbf{u}_1 , T(\mathbf{u}_2) \rangle = \langle \mathbf{u}_1 , E(\mathbf{u}_2) \rangle $$ Thus, by the inductive hypothesis, $E$ has $n$ orthonormal eigenvectors that span $U^\perp$ and because $$ E(\mathbf{x}) = \omega \mathbf{x} = T(\mathbf{x}) $$ where $(\omega, \mathbf{x})$ is an eigenpair of $E$, then all eigenvectors of $E$ are also eigenvectors of $T$, so $T$ admits a spectral decomposition $\, \blacksquare$

Now I'd like to show this by using matrices.

Attempt to prove with matrix math


Base case. Let $A = A^\top \in \mathbb{R}^{1 \times 1}$. $A$ has a real eigenvalue $\lambda \in \mathbb{R}$ and eigenvector $\mathbf{v} \in \mathbb{R}$, and $\frac{\mathbf{v}}{\| \mathbf{v} \|}$ spans $\mathbb{R} $

Inductive step. Let $A = A^\top \in \mathbb{R}^{(n + 1) \times (n + 1)}$. Again $$ \exists \lambda \in \mathbb{R} \exists \mathbf{v} \in \mathbb{R}^{n+1} \; | \; A \mathbf{v} = \lambda \mathbf{v} $$ because the fundamental theorem of algebra tells us that the characteristic polynomial of $A$ will have $n+1$ roots counted with algebraic multiplicity, and $A$'s symmetry tells us that the root(s) are real, and let $(\lambda, \mathbf{v})$ be an eigenpair of $A$. Let $U = \text{span}\{ \mathbf{v} \}$ and thus $$ U^\perp = \{ \mathbf{u} \in \mathbb{R}^{n+1} \; | \; \langle \mathbf{u}, \mathbf{v} \rangle = \mathbf{u} \cdot \mathbf{v} = \mathbf{u}^\top \mathbf{v} = 0 \} $$ Since $A$ is symmetric, $U^\perp$ is invariant under $A$: $$ A \mathbf{u} \cdot \mathbf{v} = (A \mathbf{u})^\top \mathbf{v} = \mathbf{u}^\top A^\top \mathbf{v} = \mathbf{u}^\top A \mathbf{v} = \mathbf{u}^\top (\lambda \mathbf{v}) = \lambda \mathbf{u}^\top \mathbf{v} = \lambda (\mathbf{u} \cdot \mathbf{v}) = 0 $$ thus $A \mathbf{u} \in U^\perp$, for all $\mathbf{u} \in U^\perp$.

Now let $\{\mathbf{y}_1 , \dots, \mathbf{y}_n \}$ be a basis of $U^\perp$, and let $Y = \left[\begin{smallmatrix}\mathbf{y}_1 & \dots & \mathbf{y}_n \end{smallmatrix}\right] \in \mathbb{R}^{(n+1) \times n}$. Note that $\dim U^\perp = n$ since $\dim U = 1$. Let $B$ be the transformation described by $A$ in the basis $\{\mathbf{y}_i \}$, defined as $$ B_{ij} = \langle A\mathbf{y}_j, \mathbf{y}_i \rangle = \sum_{k=1}^n AY_{jk}Y_{ki} = Y^\top A Y $$ that is, the $ij$-th entry of $B \in \mathbb{R}^{n \times n}$ is the $i$-th coordinate (w.r.t. $\{ \mathbf{y}_i \}$) of the transformed $\mathbf{y}_j$. The punchline is $$ B^\top = (Y^\top AY)^\top = Y^\top A^\top Y $$ so $B$ is symmetric because $A$ is symmetric.

Thus, by the inductive hypothesis, $B$ admits an orthonormal eigenbasis of $\mathbb{R}^n$.

The point where I'm stuck


At this point, the proofs I've read usually say something along the lines of:

Since $B$ is the transformation $A$ in the basis $\{ \mathbf{y}_i \}$, then all eigenvectors of $B$ are also eigenvectors of $A$, and further, these eigenvectors are orthogonal to $\mathbf{v}$, thus $A$ admits an orthonormal eigenbasis.

Which makes perfect sense; however, since we know this is true we should be able to show the fact that the eigenvectors of $B$ are also eigenvectors of $A$ using just matrices (i.e. the math should work out).

Attempt

My attempt was to let $(\mu, \mathbf{x})$ be an eigenpair of $B$. Then $$ B \mathbf{x} = \mu \mathbf{x} = Y^\top A Y \mathbf{x} $$ So $Y \mathbf{x}$ is $\mathbf{x}$ in the standard basis of $\mathbb{R}^{n+1}$, so I'll nickname it $\mathbf{m}$, so this gives us $$ Y B \mathbf{x} = Y \mu \mathbf{x} = \mu \mathbf{m} = Y Y^\top A \mathbf{m} $$ at this point, I wanted to say something about $Y Y^\top$, as in if $Y$ is an orthogonal matrix then we have the desired result, but it isn't since $Y \in \mathbb{R}^{(n+1) \times n}$, so it can't be.

My question/TLDR


My question written out explicitly is:

If $A \in \mathbb{R}^{(n+1)\times (n+1)}$ and $B \in \mathbb{R}^{n \times n} $ are both symmetric, and $B = Y^\top A Y$, prove that all eigenvectors of $B$ are also eigenvectors of $A$, i.e. show that if
$$ B \mathbf{x} = \mu \mathbf{x} $$ and $Y \mathbf{x} = \mathbf{m}$, then $$ A \mathbf{m} = \mu \mathbf{m} $$

$\endgroup$
1
  • $\begingroup$ What does it mean for a non-square matrix to be symmetric? $\endgroup$
    – GReyes
    Jun 7 at 23:10

2 Answers 2

1
$\begingroup$

The piece of data you are missing is that, because $y_1,\ldots,y_n$ is an orthonormal basis for $U^\perp$, then $YY^\top$ is the orthogonal projection onto $U^\perp$. Thus from $B=Y^\top AY$ and $Bx=\mu x$ you get, since $Yx\in U^\perp$, $$ A(Yx)=YY^\top AYx=YBx=Y(\mu x)=\mu(Yx). $$

$\endgroup$
0
$\begingroup$

I think I've got it:

Another requirement/something that is worth emphasizing: $\{ \mathbf{y}_i \}$ is an orthonormal basis of $U^\perp$, so \begin{gather} Y^\top Y = I \end{gather} since \begin{gather} (Y^\top Y)_{ij} = \sum_k Y^\top_{ik} Y_{kj} = \sum_k Y_{ki} Y_{kj} = \langle \mathbf{y}_i, \mathbf{y}_j \rangle = \mathbf{y}_i \cdot \mathbf{y}_j = \delta_{ij} \end{gather} Then, since $Y \mathbf{x} = \mathbf{m}$ \begin{gather} Y^\top \mathbf{m} = \mathbf{x} \end{gather} then \begin{gather} B \mathbf{x} = \mu \mathbf{x} \xrightarrow[B = Y^\top A Y]{} Y^\top A Y \mathbf{x} = \mu \mathbf{x} \end{gather} Now replace $\mathbf{x}$ with $Y^\top \mathbf{m}$, and replace $Y \mathbf{x}$ with $\mathbf{m}$ \begin{gather} Y^\top A \mathbf{m} = \mu Y^\top \mathbf{m} \to Y^\top (A\mathbf{m} - \mu \mathbf{m}) = \mathbf{0} \end{gather} Which means that $A\mathbf{m} - \mu \mathbf{m}$ is either orthogonal to $U^\perp$, or $A\mathbf{m} - \mu \mathbf{m}= \mathbf{0}$. Since $\text{im } Y = U^\perp$, $\mathbf{m} \in U^\top$, and since $U^\top$ is invariant under $A$, then $A\mathbf{m} - \mu \mathbf{m} \in U^\perp$ and thus $$ A\mathbf{m} - \mu \mathbf{m} = \mathbf{0} \quad \blacksquare $$

So we can take $B$'s orthonormal eigenbasis, $\{ \mathbf{x}_i \}$, and let $$ X = \begin{bmatrix} \mathbf{x}_1 & \dots & \mathbf{x}_n \end{bmatrix} $$ then $$ YX = \begin{bmatrix} Y\mathbf{x}_1 & \dots & Y\mathbf{x}_n \end{bmatrix} $$ are all eigenvectors of $A$, and the eigenvectors are all orthogonal since the eigenvalues are real. So if we take all of those eigenvectors of $A$ with the original eigenvector $\mathbf{v}$ (and normalize), we get the orthonormal eigenbasis of $A$: $$ \{ Y\mathbf{x}_i \} \cup \{ \mathbf{v} \} $$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .