# Proof of a linear algebra lemma for Cohn-Vossen's theorem

For the proof of Cohn-Vossen's rigidity theorem I need to prove the next lemma (can be found in Montiel-Ros's Curves and Surfaces page 218):

If $$\Phi$$ and $$\Psi$$ are two definite self-adjoint endomorphisms of a Euclidean vector plane and $$det \Phi = det \Psi$$. Then, $$det (\Phi + \Psi) \leq 0$$ and that equality occurs if and only if $$\Phi = - \Psi$$.

The proof says the following:

If $$det(\Phi +\Psi) > 0$$, the endomorphism $$\Phi+\Psi$$ would be definite, say, positive definite. Now, we take $$\{ e_1, e_2 \}$$ a basis of the plane diagonalizing $$\Phi$$, one has $$\langle \Phi(e_i), e_i \rangle + \langle \Psi(e_i), e_i \rangle > 0, \;\;\;\; i = 1,2.$$ Consequently, $$det \Phi = \langle \Phi(e_1), e_1 \rangle \langle \Phi(e_2), e_2 \rangle \stackrel{(1)}{>} \langle \Psi(e_1), e_1 \rangle \langle \Psi(e_2), e_2 \rangle \geq \langle \Psi(e_1), e_1 \rangle \langle \Psi(e_2), e_2 \rangle - \langle \Psi(e_1), e_2 \rangle^2 \stackrel{(2)}{=} det \Psi$$ which gives a contradiction to our hypothesis. Therefore, $$det( \Phi + \Psi ) \leq 0$$.

Here, I don't understand inequality (1) and equality (2).

Moreover, the lemma says the following:

In the above inequality, equality occurs if and only if $$\Phi = - \Psi$$.

The proof says the following:

Following the reasoning above, if equality holds, there wouldbe at least a non-null vector in the kernel of $$\Phi + \Psi$$. Let $$\{u_1, u_2 \}$$ be a basis diagonalizing $$\Phi + \Psi$$, that is, such that $$\Phi(u_1) + \Psi(u_1) = 0 \;\;\;\; and \;\;\;\; \Phi(u_2) + \Psi(u_2) = \lambda u_2, \;\;\; \lambda \in \mathbb{R}.$$ From the first equality we deduce that $$\langle \Phi(u_1), u_1 \rangle = - \langle \Psi(u_1), u_1 \rangle \;\;\;\; and \;\;\;\; \langle \Phi(u_1), u_2 \rangle = - \langle \Psi(u_1), u_2 \rangle,$$ which together with the facts that $$det \Phi = det \Psi$$ and that $$\Phi$$ and $$\Psi$$ are definite, gives the equality $$\langle \Phi(u_2), u_2 \rangle \stackrel{(3)}{=} - \langle \Psi(u_2), u_2 \rangle,$$ implying $$\lambda = 0$$. Thus, in this case, $$\Phi = - \Psi$$.

Of this latest part I don't understand the equality (3).

Could you show me why these equations hold?

• @TedShifrin great! I could figure out (2) with your comment! Oct 5, 2018 at 21:42
• In addition to Ted's comment, notice that the lemma is trivially false if $\Phi$ and $\Psi$ are both the identity. Oct 5, 2018 at 22:59
• for a proof of the theorem you can look at page 87 : math.brown.edu/~deigen/chern.pdf Oct 7, 2018 at 17:06

Once I fixed a crucial typo, some things became clearer. Since we're assuming $$\Phi+\Psi$$ is positive definite, we have $$\langle (\Phi+\Psi)e_i,e_i\rangle > 0$$ for $$i=1,2$$, so $$\langle\Phi(e_i),e_i\rangle > -\langle\Psi(e_i),e_i\rangle$$ for $$i=1,2$$. "Therefore," so to speak, $$\langle\Phi(e_1)e_1\rangle\langle\Phi(e_2),e_2\rangle>\langle\Psi(e_1),e_1\rangle\langle\Psi(e_2),e_2\rangle,$$ establishing inequality (1). This is the proof the authors intended, but of course it's wrong unless we are assuming $$\Psi$$ is negative definite here, so that both right-hand sides are positive and we can multiply the inequalities.

Equality (2) is just the computation of $$\det\Psi$$ using a matrix representation with respect to the basis $$\{e_1,e_2\}$$.

Now that we've narrowed things down to assuming that $$\Phi$$ is positive definite and $$\Psi$$ is negative definite, let's look at (3). This is also following from a determinant computation: \begin{align*} \det\Phi&=\langle \Phi(u_1),u_1\rangle \langle \Phi(u_2),u_2\rangle - \langle \Phi(u_1),u_2\rangle^2\\ \det\Psi &= \langle \Psi(u_1),u_1\rangle\langle \Psi(u_2),u_2\rangle -\langle \Psi(u_1),u_2\rangle^2. \end{align*} Since $$\det\Phi=\det\Psi$$, substituting the first two equalities, we get $$\langle \Phi(u_1),u_1\rangle \langle \Phi(u_2),u_2\rangle - \langle \Phi(u_1),u_2\rangle^2 = -\langle \Psi(u_1),u_1\rangle \langle \Phi(u_2),u_2\rangle - \langle \Psi(u_1),u_2\rangle^2 = \langle \Psi(u_1),u_1\rangle\langle \Psi(u_2),u_2\rangle -\langle \Psi(u_1),u_2\rangle^2,$$ and so $$\langle\Psi(u_2),u_2\rangle = -\langle \Phi(u_2),u_2\rangle$$, as they claimed.

Let me reiterate that the lemma is false as stated. If we assume both $$\Phi$$ and $$\Psi$$ are positive definite, then we in fact should conclude that $$\det(\Phi-\Psi)\le 0$$ with equality holding iff $$\Phi=\Psi$$. If we assume (as apparently these authors meant to) that $$\Phi$$ is positive definite and $$\Psi$$ is negative definite, then we conclude that $$\det(\Phi+\Psi)\le 0$$ with equality holding iff $$\Phi=-\Psi$$.

• @EvaMGG: Sorry about my own typos. Generally, rather than editing, it's better just to put a comment to get it clarified. (I mistakenly rejected the edit because I couldn't believe I had it wrong!! :P) ... Anyhow, I hope the situation is now clear. P.S. I think the book's notation is a bit cumbersome. I would have just used matrices and simpler letters :P Oct 6, 2018 at 17:06
• Don't worry, it's everything ok :) Oct 6, 2018 at 17:12

I just had a quick chat with Prof. Montiel and it seems to be a typo in the text. Essentially, you should substitute $$det (\Phi + \Psi)$$ by $$det (\Phi - \Psi)$$.

This was the change that one could spot in the proof by Chern (page 87) that I pointed out in the comments. The difference between the proofs amounts to the fact that Montiel-Ros was taken from an article he didn't remember (he mentionned two mathematicians: the Spanish Santaló and a German called something like "Fechlossen").

I think that reusing the valuable answer by @TedShifrin you can now reconstruct what the lemma should look like and more importantly, you will be able to use it in the proof of Cohn-Vossen's theorem.

Let me know if this works out for the proof.