The other day I stumbled upon LittleO's answer in this question. To make reading it easier I did a straight copy-and-paste here:

Here's some intuition (but not a rigorous proof).

If $A$ is hermitian (with entries in $\mathbb C$), you can easily show that the eigenvalues of $A$ are real and that eigenvectors corresponding to distinct eigenvalues are orthogonal.

Typically, all the eigenvalues of $A$ are distinct. (It is in some sense a huge coincidence if two eigenvalues turn out to be equal.) So, typically $A$ has an orthonormal basis of eigenvectors.

Even if $A$ has some repeated eigenvalues, perturbing $A$ slightly will probably cause the eigenvalues to become distinct, in which case there is an orthonormal basis of eigenvectors. By thinking of $A$ as a limit of slight perturbations of $A$, each of which has an ON basis of eigenvectors, it seems plausible that $A$ also has an ON basis of eigenvectors.

While this idea struck me as very ingenious and intuitive, I'm really having difficulty formalising it into a real proof. The major obstacles are:
1). How to perturb an hermitian $A$ while keeping its eigenvalues distinct? In particular, how to find an hermitian sequence $A_n$ that have distinct eigenvalues and converge to $A$ in some norm?
2). Based on 1), how to continuously associate the eigenvector family $V_n:=[v_{n,j},j=1,\cdots,d]$ (corresponding to $A_n$, assuming $A_n$ are $d$ by $d$) to $A_n$? Having solved this I would have $V:=[v_j,j=1,\cdots,d]$ (corresponding to $A$) are the limits of $\{v_{n,j},j=1,\cdots,d\}$ respectively and hence $V^HV = \lim_{n\to\infty} V_n^HV_n = I$ and we are done.

  • 1
    $\begingroup$ Note that in general, we should not expect the limit of diagonalizable matrices to be diagonalizable. In particular, note that $$ \pmatrix{1 + 1/n & 1\\0 & 1} $$ is diagonalizable for all $n \in \Bbb N$, but its limit is not diagonalizable. $\endgroup$ – Omnomnomnom Sep 24 '17 at 20:53

Let me answer part $(1)$ as part $(2)$ was answered by Omnomnomnom. Consider the space $W$ of Hermitian $n \times n$ matrices as a real vector space (which is isomorphic to $\mathbb{R}^{n^2}$) and let $D \colon W \rightarrow \mathbb{R}$ be the map which maps a Hermitian matrix $A$ to the discriminant of the characteristic polynomial of $A$. The map $D$ is polynomial in the entries of the matrix $A$ and $D(A) \neq 0$ if and only if the roots of the characteristic polynomial of $A$ are distinct. For example, if $n = 2$, a Hermitian matrix is described as

$$ A = \begin{pmatrix} x & a + ib \\ a - ib & y \end{pmatrix} $$

for $a,b,x,y \in \mathbb{R}$ and

$$ D(A) = (x + y)^2 - 4(xy - |a + ib|^2) = (x - y)^2 + (2a)^2 + (2b)^2. $$

Since $D$ is a non-zero polynomial (it doesn't vanish on the diagonal matrices with distinct eigenvalues), the set $\{ A \in W \, | \, D(A) \neq 0 \}$ is dense in $W$ so any Hermitian matrix in $D^{-1}(0)$ can approximated by Hermitian matrices with distinct eigenvalues.

In fact, one can show that the set $D^{-1}(0)$ of Hermitian matrices with a repeated eigenvalue is of codimension three in $W$ (see "avoidance of crossings" in Lax's Linear Algebra) but I don't know how to do it without the spectral theorem.

  • $\begingroup$ Thanks for your answer! Density is sufficient here. $\endgroup$ – Vim Sep 25 '17 at 11:19
  • $\begingroup$ I think the space of hermitians should be $n(n-1)/2$-dimensional instead. $\endgroup$ – Vim Sep 25 '17 at 12:46
  • $\begingroup$ @Vim: You can see that your dimension count is false already for $n = 2$. A Hermitian matrix depends on $n$ real parameters (diagonal entries $a_{ii}$) and $1 + 2 + \dots + (n-1) = \frac{n(n-1)}{2}$ complex parameters $a_{ij}$ for $i > j$ which results in a total of $n + 2 \frac{n(n-1)}{2} = n^2$ real parameters. $\endgroup$ – levap Sep 25 '17 at 13:23
  • $\begingroup$ alrite I was thinking over $\Bbb C$... but still a silly mistake. $\endgroup$ – Vim Sep 25 '17 at 13:25
  • $\begingroup$ Note that it is not a vector space over $\mathbb{C}$. $\endgroup$ – levap Sep 25 '17 at 13:25

In the below, I use $A^*$ to denote the conjugate transpose of $A$.

In truth, I'm not sure how to answer part 1. In particular, I'm finding it difficult to prove the existence of such a perturbation in such a fashion that doesn't amount to what is essentially a full proof of the spectral theorem.

It is notable, however, that if $\lambda,v$ is an eigenpair of $A$ and $\|v\| = 1$, then the matrix $B = A + \epsilon vv^*$ (where $\epsilon > 0$ can be made arbitrarily small) has "all the same eigenvectors" and will satisfy

  • $Bu = Au$ for $u \perp v$
  • $Av = (\lambda + \epsilon)v$

Moreover, it is straightforward to show that $A(vv^*) = (vv^*)A$.

Assuming that part 1 is accounted for, there is a neat argument for part 2. Presumably, we have a sequence of Hermitian matrices $A_n \to A$ such that each $A_n$ has distinct eigenvalues, and we have an associated sequence of unitary matrices $U_n$ such that $U_n^*A_nU_n$ is a diagonal matrix.

Notably, the set of unitary matrices is compact in $\Bbb C^{d \times d}$.

So, there exists a convergent subsequence $U_{n_k}$. Let $U$ denote the limit of this subsequence. Notably, we have $A_{n_k} \to A$, since the subsequence of a convergent sequence is convergent with the same limit. From there, we note that matrix-multiplication and the $A \mapsto A^*$ operator are continuous to conclude that $$ U^*AU = \left( \lim_{k \to \infty} U_{n_k}\right)^* \left( \lim_{k \to \infty} A_{n_k}\right) \left( \lim_{k \to \infty} U_{n_k}\right) \\ = \left( \lim_{k \to \infty} U_{n_k}^*\right) \left( \lim_{k \to \infty} A_{n_k}\right) \left( \lim_{k \to \infty} U_{n_k}\right) \\ = \lim_{k \to \infty} (U_{n_k}^*A_{n_k}U_{n_k}) $$ So, $U^*AU$ is the limit of a sequence of diagonal matrices in $\Bbb C^{d\times d}$. Since the set of diagonal matrices is closed in $\Bbb C^{d \times d}$, we may conclude that this limit is also diagonal. In other words, the Hermitian matrix $A$ is diagonalized by the unitary matrix $U$.

  • 2
    $\begingroup$ Even slicker: if $D_{n,B}$ is the set of diagonal matrices with entries bounded by $B$, then $D_{n,B}\times U_n$ is compact, and thus the set of all $U^*DU$ is closed since it's the image of a comapct set under a continuous function. $\endgroup$ – Hurkyl Sep 24 '17 at 22:15
  • $\begingroup$ Thanks for your answer. To clarify a detail: when you are talking about $\Bbb C^{d\times d}$, what's the topology you're using here? I would assume it is the topology induced by Frobenius norm (i.e. "Entrywise" topology). $\endgroup$ – Vim Sep 25 '17 at 0:01
  • $\begingroup$ @Vim That works. As it turns out, every normed topology on $\Bbb C^{d \times d}$ is equivalent to the "entrywise" topology. $\endgroup$ – Omnomnomnom Sep 25 '17 at 0:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.