24
$\begingroup$

The problem statement is as follows:

Prove that for a normal matrix $A$, eigenvectors corresponding to different eigenvalues are necessarily orthogonal.

I can certainly prove that this is the case, using the spectral theorem. The gist of my proof is presented below.

If possible, I would like to find a simpler proof. I was hoping that there might be some sort of manipulation along these lines, noting that $$ \langle Av_1,A v_2\rangle = \langle v_1,A^*Av_2\rangle = \langle v_1,AA^*v_2\rangle = \langle A^* v_1,A^* v_2 \rangle $$

Any ideas here would be appreciated.


My proof:

Let $\{v_{\lambda,i}\}$ be an orthonormal basis of eigenvectors (as guaranteed by the spectral theorem) such that $$ A v_{\lambda,i} = \lambda v_{\lambda,i} $$ Let $v_1,\lambda_1$ and $v_2,\lambda_2$ be eigenpairs with $\lambda_1 \neq \lambda_2$. We may write $ v_1 = \sum_{i,\lambda}a_{i,\lambda}v_{i,\lambda} .$ We then have $$ 0 = Av_1 - \lambda_1 v_1 = \sum_{i,\lambda}(\lambda - \lambda_1)a_{i,\lambda}v_{i,\lambda} $$ So that $a_{i,\lambda} = 0$ when $\lambda \neq \lambda_1$. Similarly, we may write $v_2 = \sum_{i,\lambda}b_{i,\lambda}v_{i,\lambda}$, and note that $b_{i,\lambda} = 0$ when $\lambda \neq \lambda_2$. From there, we have $$ \langle v_1,v_2 \rangle = \sum_{i,\lambda}a_{i,\lambda}b_{i,\lambda} $$ the above must be zero since for each pair $i,\lambda$, either $a_{i,\lambda}=0$ or $b_{i,\lambda} = 0$.

$\endgroup$
2
  • $\begingroup$ "Let $v_{\lambda,i}$ be an orthonormal basis of eigenvectors..." I'd guess that this might be something very close to what would need to be proved. $\endgroup$ May 3, 2014 at 2:13
  • $\begingroup$ @PavelJiranek I was worried that I had used circular logic at some point. However, the existence of such a basis (i.e. the spectral theorem) comes directly from the Schur triangularization theorem, which says nothing about normal matrices in particular. $\endgroup$ May 3, 2014 at 19:11

7 Answers 7

20
$\begingroup$

Assume $\;\lambda\neq \mu\;$ and

$$\begin{cases}Av=\lambda v\;\,\implies\; A^*v=\overline \lambda v\\{}\\Aw=\mu w\implies A^*w=\overline\mu w\end{cases}$$

From this we get:

$$\begin{cases}\langle v,Aw\rangle=\langle v,\mu w\rangle=\overline\mu\langle v,w\rangle\\{}\\ \langle v,Aw\rangle=\langle A^*v,w\rangle=\langle\overline\lambda v,w\rangle=\overline\lambda\langle v,w\rangle \end{cases}$$

and since $\;\overline\mu\neq\overline\lambda\;$ , we get $\;\langle v,w\rangle =0\;$

Question: Where did we use normality in the above?

$\endgroup$
11
  • 3
    $\begingroup$ How de we know that $v$ is also an eigenvector for $A^*$? $\endgroup$
    – Berci
    May 2, 2014 at 22:55
  • 1
    $\begingroup$ From the other two answers this follows, of course, using that $A$ is normal. But can you show it directly? $\endgroup$
    – Berci
    May 2, 2014 at 22:57
  • 3
    $\begingroup$ Can someone give me a clue as to where you used normality in the above? $\endgroup$
    – ikoikoia
    Jul 9, 2018 at 1:19
  • 2
    $\begingroup$ If p(x,y) is a polynomial then since A is normal, the matrix B := p(A,A*) is also normal. Normality of B implies that B and B* have the same kernel, using the identity <Bv,Bv> = <B* v,B* v> derived by the OP. By taking p(x,y) := x-lambda we get the implications asserted in the answer above. $\endgroup$ Jun 20, 2019 at 15:14
  • 1
    $\begingroup$ Check this answer $\endgroup$ Dec 7, 2020 at 18:08
13
$\begingroup$

Specializing your identity to $v_1=v_2=v$, we get $\|Av\|=\|A^*v\|$. Hence $\ker A=\ker A^*$. Recalling that $\ker A^* = (\operatorname{ran} A)^\perp$ for general $A$, we conclude that the kernel and range of a normal matrix are mutually orthogonal.

It remains to apply the above conclusion to $A-\lambda I$ where $\lambda$ is an eigenvalue of $A$.

$\endgroup$
1
  • $\begingroup$ Neat! Although DonAntonio's proof is more along the lines I was thinking of, this is a nice perspective. $\endgroup$ May 2, 2014 at 23:16
12
$\begingroup$

The answers to this question are very good but they are glossing over the fact that if $A \in \mathbb C^{n \times n}$ is normal then $Ax = \lambda x \implies A^*x = \bar \lambda x$. I think it's helpful to just spell out the whole thing explicitly.

Lemma: If $M \in \mathbb C^{n \times n}$ is normal, then $M$ and $M^*$ have the same null space.

Proof: Let $M \in \mathbb C^{n \times n}$ be a normal matrix. Then \begin{align} &M^*x = 0 \\ \iff & \| M^* x \|^2 = 0 \\ \iff & \langle M^*x, M^* x \rangle = 0 \\ \iff & \langle M M^* x, x \rangle = 0 \\ \iff & \langle M^* M x, x \rangle = 0 \\ \iff & \langle Mx, Mx \rangle = 0 \\ \iff & \| Mx \|^2 = 0 \\ \iff & Mx = 0. \end{align}

Lemma: If $A \in \mathbb C^{n \times n}$ is normal then $Ax = \lambda x \implies A^* x = \bar \lambda x$.

Proof: Suppose that $A \in \mathbb C^{n \times n}$ is normal and $Ax = \lambda x$. So $x$ is a null vector of $M = A - \lambda I$. Note that $M$ is normal, as you can check by expanding both $M^* M$ and $M M^*$. By the above lemma, $x$ is also a null vector of $M^* = A^* - \bar \lambda I$. Thus $A^*x = \bar \lambda x$.

Theorem: If $x$ and $y$ are eigenvectors corresponding to distinct eigenvalues of a normal matrix $A \in \mathbb C^{n \times n}$, then $\langle x, y \rangle = 0$.

Proof: Let $A \in \mathbb C^{n \times n}$ be a normal matrix, and suppose that $x$ and $y$ are eigenvectors of $A$ corresponding to distinct eigenvalues $\lambda$ and $\gamma$, respectively. Note that $$\langle x, Ay \rangle = \langle x, \gamma y \rangle = \bar \gamma \langle x, y \rangle.$$ On the other hand, $$ \langle x, Ay \rangle = \langle A^* x, y \rangle = \langle \bar \lambda x, y \rangle = \bar \lambda \langle x, y \rangle. $$ So we find that $\bar \gamma \langle x, y \rangle = \bar \lambda \langle x, y \rangle$, which implies that $\langle x, y \rangle = 0$.

$\endgroup$
6
  • $\begingroup$ Thank you for this very helpful answer! Shouldn't the last member of the last two displayed expressions have unconjugated gamma and lambda? $\endgroup$ May 1, 2022 at 22:57
  • 1
    $\begingroup$ @RodolfoOviedo I’m using the convention that if $x, y \in \mathbb C^n$ then $\langle x, y \rangle = \sum_i x_i \bar{y_i}$. With this convention, I think the way that I wrote it is correct. Sometimes people (especially physicists, I believe) define $\langle x, y \rangle$ to be $\sum_i \bar{x_i} y_i$. Perhaps you have that definition in mind. $\endgroup$
    – littleO
    May 2, 2022 at 0:14
  • $\begingroup$ Thanks for your answer! After the second sentence of the proof of the second lemma, I would add: Because $A$ is normal, so is $M$, as you can check by expanding both $M^*M$ and $MM^*$. $\endgroup$ May 2, 2022 at 22:45
  • $\begingroup$ Am I right? If so, should you or I edit the answer? $\endgroup$ May 2, 2022 at 22:52
  • $\begingroup$ @RodolfoOviedo I just made the change you suggested, thanks. $\endgroup$
    – littleO
    May 2, 2022 at 23:39
4
$\begingroup$

linear_algebra_done_right This is from linear algebra done right. btw, it's the greatest book about linear algebra I've ever seen!


Proof: Suppose $\alpha,\beta$ are distinct eigenvalues of $T$, with corresponding eigenvectors $u,v$. Thus, $Tu = \alpha u$ and $Tv = \beta v$. From 7.21 we have $T^*v = \bar \beta v.$ Thus, \begin{align} (\alpha - \beta)\langle u,v \rangle &= \langle \alpha u,v \rangle - \langle u, \bar \beta v \rangle \\ &= \langle Tu,v \rangle - \langle u, T^*v \rangle \\ & = 0. \end{align} Because $\alpha \neq \beta$, the equation above implies $\langle u,v \rangle = 0$. Thus, $u$ and $v$ are orthogonal, as desired.

$\endgroup$
1
  • $\begingroup$ LADR should have been the first place that I looked. Thanks for your answer $\endgroup$ Nov 24, 2020 at 14:22
3
$\begingroup$

I try to give another simple proof to $$T^*v=\bar{\lambda}v ~\text{ if }~ Tv=\lambda v$$ where $T$ is a normal operator on a Hilbert space $H$.

Suppose $V=\ker(T-\lambda I)$. Since $T^*$ communicate with $T$, $$T^*V\subset V.$$ Because $$\langle v,T^*v\rangle =\langle Tv,v\rangle =\langle \lambda v,v\rangle=\langle v,\bar{\lambda}v\rangle ~~\forall v \in V, $$ $\langle u,T^*v\rangle =\langle u, \bar{\lambda}v\rangle ~\forall u,v\in V$ by polarisation identity, and thus $T^*v=\bar{\lambda}v.$

REMARK: Let $\sigma:V\times V\to W$ be a sesquilinear form, where $V$ and $W$ are linear vector spaces over $\mathbb{C}$. The follwing formula is called Polarisation Identity : $$\sigma(u,v)=\sum_{k=0}^3 i^k\sigma(u+i^k v, u+i^kv). $$

$\endgroup$
4
  • $\begingroup$ I don't really see how this answer addresses the question being asked. Also, the question being asked was answered 3 years ago. $\endgroup$ Jun 12, 2017 at 15:01
  • $\begingroup$ @Omnomnomnom This is why $Av=\lambda v\Rightarrow A^*v=\bar{\lambda}v$ using no spectral theorem which is the point to the question I think. $\endgroup$
    – C. Ding
    Jun 13, 2017 at 2:37
  • $\begingroup$ That is not the point to the question, actually. The point is to show that eigenvectors corresponding to different eigenvalues are necessarily orthogonal. $\endgroup$ Jun 13, 2017 at 2:58
  • 1
    $\begingroup$ Oh, sorry to disturb you. As I give some additional remarks to the right answer you have accepted. $\endgroup$
    – C. Ding
    Jun 13, 2017 at 3:04
1
$\begingroup$

I'll use the notation from Introduction to Linear Algebra by Strang. Let $A^H$ be the transpose conjugate of $A$, and suppose $A$ is normal, i.e. $AA^H = A^H A$.

Examine an eigenpair $Ax = \lambda x$. Multiply by $A^H$ and we have $A^H Ax = AA^H x = \lambda A^H x$. Assume only one $x$ with eigenvalue $\lambda$ (see note at end), so here we know that $A^H x$ is a multiple of $x$, i.e. $A^H x = c x$. Multiply by $x^H$ and we have $c = \bar \lambda$, i.e. the conjugate of $\lambda$.

Now consider another eigenpair $A y = \mu y$ with $\mu \neq \lambda$. From the result in the previous paragraph, we have $y^H A = \mu y^H$. Now multiply by $x$ and we obtain $\mu y^H x = \lambda y^H x$, which leads to $y^H x = 0$.

Note: if there are multiple $x$ with eigenvalue $\lambda$, then choose $x$ to be the linear combination that is also an eigenvector of $A^H$. Such a choice is always be possible given that there's at least one eigenvector with eigenvalue $\lambda$ (proof is left as an exercise for the reader).

$\endgroup$
-1
$\begingroup$

A normal matrix is unitarily similar to diagonal matrix.

$$A = UDU^{-1}$$ where $U$ is Unitary matrix.

Eigen decompositions tells that $U$ is a matrix composed of columns which are eigenvectors of $A$. And matrix $D$ is Diagonal matrix with eigenvalues on diagonal.

Property: Columns of Unitary matrix are orthogonal.

So, columns of $U$ (which are eigenvectors of $A$) are orthogonal.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .