# Invariant Subspace of Two Linear Involutions

I'd love some help with this practice qualifier problem:

If $A$ and $B$ are two linear operators on a finite dimensional complex vector space $V$ such that $A^2=B^2=I$ then show that $V$ has a one or two dimensional subspace invariant under $A$ and $B$.

Thanks!

-
Your argument doesn't seem to be valid; what if the eigenspaces corresponding to $1$ are empty in both cases (e.g. if $A = B = -I$)? It is a relatively small fix from here though. – Qiaochu Yuan Apr 6 '11 at 17:07
@JBeardz Why are A and B diagonalizable? (Sorry, I'm learning Linear Algebra). Also, I would tweak your argument a bit: if the eigenspaces corresponding to 1 for A and ditto for B do not intersect, then the corresponding ones to -1 must. Would you agree? – Weltschmerz Apr 6 '11 at 17:16
@Weltschmerz: Since $A^2 = I$, then $A$ satisfies the polynomial $t^2-1 = (t-1)(t+1)$. That means that the minimal polynomial splits and is squarefree (unless $\mathrm{char}(F)=2$); this is necessary and sufficient for a linear operator to be diagonalizable. – Arturo Magidin Apr 6 '11 at 17:24
@Arturo: Thanks! – Weltschmerz Apr 6 '11 at 17:31
I've added a bounty, for a full, clear answer. – Jon Beardsley Apr 9 '11 at 20:38

Note: First couple of paragraphs relate to some statements made in a previous version of the question.

If the two operators are the identity, then it certainly has a subspace which is both invariant under both, and $1$ dimensional. The phrasing of the question does not suggest that there are no larger subspaces which are invariant under both (after all, $\mathbf{V}$ itself is always invariant under both $A$ and $B$, so you always have larger dimensional invariant subspaces. In fact, what you want to find is the smallest possible nontrivial subspace which is invariant under both $A$ and $B$.

The reason you want this is that if you can find a $1$-dimensional subspace that is invariant under an operator, that subspace gives you an eigenvector and an eigenvalue. If you can find a $1$-dimensional subspace that is invariant under both $A$ and $B$, then this tells you that there is a vector which is an eigenvector for both $A$ and $B$, which is a very useful condition (note that the same vector may be an eigenvector for both $A$ and $B$, but corresponding to different eigenvalues). Unfortunately, you don't always have this, which is why the problem says that sometimes you need to take a $2$-dimensional subspace instead.

For example, if you take $A$ to be the linear transformation on $\mathbb{C}^2$ that sends $(1,0)$ to $(-1,0)$ and $(0,1)$ to $(0,1)$; and you take $B$ to be the linear transformation that sends $(1,0)$ to $(0,-1)$ and $(0,1)$ to $(-1,0)$, then $A$ and $B$ have no common eigenvector, so you will not be able to find a $1$-dimensional subspace that is invariant under both $A$ and $B$.

You are correct that the only possible eigenvalues of $A$ and of $B$ are $1$ and $-1$; in fact, more can be said: unless the characteristic of the field is $2$, then $A$ and $B$ are diagonalizable, because their minimal polynomials divide $t^2-1$, which splits and is square free.

Let $E^A_{1}$ and $E^A_{-1}$ be the eigenspaces of $A$ associated to $1$ and $-1$, respectively (one of them could be trivial, for example if $A=I$ then $E^A_{-1}=\{\mathbf{0}\}$, and similarly if $A=-I$; that's fine); and similarly with $E^B_{1}$ and $E^B_{-1}$. If $E^A_{\lambda}\cap E^B_{\mu}\neq\{\mathbf{0}\}$ for some $\lambda,\mu\in\{1,-1\}$, then you'll be done: their intersection contains a $1$-dimensional subspace that is invariant for both $A$ and $B$.

So you are on the right track in trying to find a nontrivial intersection of eigenspaces.

Unfortunately, you cannot assert that you will necessarily have $E^A_{1}$ and $E^B_{1}$ intersecting nontrivial (what if $A=-I$ and $B=I$, for example? Then $E^A_{1}={\mathbf{0}}$).

Since $A$ and $B$ have to be diagonalizable, you know that $\dim(\mathbf{V}) = \dim(E^A_1)+\dim(E^A_{-1}) = \dim(E^B_1)+\dim(E^B_{-1})$. So try to figure out what would happen if all four intersections are trivial (we know it can happen, as I already gave an example above where it happens).

Added. Okay, let's finish this, with the assumption that $\mathbf{V}$ is a complex vector space.

If $E^A_{i}\cap E^B_{j}\neq\{\mathbf{0}\}$ for some $i,j\in\{1,-1\}$, then we are done: just take a vector which is an eigenvector of both $A$ and $B$, and this gives a one dimensional invariant subspace. So we may assume that $E^A_{i}\cap E^B_{j}=\{\mathbf{0}\}$, $i,j\in\{1,-1\}$. That is, we must be in the situation:

$\dim(\mathbf{V})= 2n$, $\dim(E^A_{i}) = \dim(E^B_j) = n$, and $E^A_i\cap E^B_{j} = \{\mathbf{0}\}$ for $i,j\in\{1,-1\}$.

(The dimensions follow because if either $E^A_i$ has dimension more than half the dimension of $\mathbf{V}$, then it must intersection one $E^B_j$ nontrivially by simple dimension considerations, and symmetrically for the $E^B_j$; so the dimension of $\mathbf{V}$ must be even, and the eigenspaces must each have dimension exactly half).

Since $A$ and $B$ are diagonalizable, we know that $$\mathbf{V}=E^A_1\oplus E^A_{-1} = E^B_1\oplus E^B_{-1}.$$ Given $\mathbf{v}\in \mathbf{V}$, we can write it uniquely as $\mathbf{v}=\mathbf{y}_1+\mathbf{y}_{-1}$, with $\mathbf{y}_j\in E^B_j$; define $p^B_{j}\colon \mathbf{V}\to E^B_{j}$ by $p^B_j(\mathbf{y}_1+\mathbf{y}_{-1}) = \mathbf{y}_j$.

The restriction of $p^B_{1}$ to $E^A_1$ is injective, since $\mathrm{ker}(p^B_1) = E^B_{-1}$; likewise for $p^B_{-1}$. Let $p^B_{1,1}$ be the restriction of $p^B_{1}$ to $E^A_1$, let $p^B_{1,-1}$ be the restriction of $p^B_{-1}$ to $E^A_{-1}$; and likewise let $p^B_{-1,1}$ be the restriction of $p^B_{1}$ to $E^A_{-1}$, and $p^B_{-1,-1}$ be the restriction of $p^B_{-1}$ to $E^A_{-1}$. Then $p^B_{i,j}\colon E^A_i\to E^B_j$ is an isomorphism, being a one-to-one linear transformation between spaces of the same dimension.

Let $T\colon E^A_1\to E^A_1$ be the operator defined by $$T = \left(p^B_{1,-1}\right)^{-1}\circ p^B_{-1,-1}\circ\left(p^B_{-1,1}\right)^{-1}\circ p^B_{1,1}.$$ Then since $E^A_1$ is a complex vector space, $T$ has an eigenvector $\mathbf{x}_1$, corresponding to some eigenvalue $\lambda$.

Let $\mathbf{x}_1 = \mathbf{y}_1 + \mathbf{y}_{-1}$, with $\mathbf{y}_j \in E^B_{j}$. Note that $\mathbf{y}_j\neq \mathbf{0}$ (if either one is zero, then we get a vector which is an eigenvector for both $A$ and $B$, and we already ruled out this case). Let $$\mathbf{x}_{-1} = (p^B_{-1,1})^{-1}(\mathbf{y}_1) = (p^B_{-1,1})^{-1}\circ p^B_{1,1}(\mathbf{x}_1)\in E^A_{-1}.$$ Then $\mathbf{x}_{-1}\neq\mathbf{0}$. Therefore $\mathbf{x}_1,\mathbf{x}_{-1}$ are linearly independent. Let $\mathbf{W}=\mathrm{span}(\mathbf{x}_1,\mathbf{x}_{-1})$. Since $\mathbf{x}_i$ is an eigenvector of $A$, that means that $\mathbf{W}$ has a basis of eigenvectors of $A$, and therefore $\mathbf{W}$ is $A$-invariant.

Write $\mathbf{x}_{-1} = \mathbf{z}_1+\mathbf{z}_{-1}$, with $\mathbf{z}_j\in E^B_j$. Then $$\mathbf{z}_1 = p^B_{-1,1}(\mathbf{x}_{-1}) = p^B_{-1,-1}\left((p^B_{-1,1})^{-1}(\mathbf{y}_1)\right) = \mathbf{y}_1.$$ And notice that since $T(\mathbf{x}_1) = \lambda\mathbf{x}_1$, then $$\mathbf{y}_{-1} = p^B_{1,-1}(\mathbf{x}_1) = \frac{1}{\lambda}p^B_{1,-1}(T(\mathbf{x}_1)) = \frac{1}{\lambda}p^B_{-1,-1}\circ (p^B_{-1,1})^{-1}\circ p^B_{1,1}(\mathbf{x}_1) = \frac{1}{\lambda}p^B_{-1,-1}(\mathbf{x}_{-1}) = \frac{1}{\lambda}\mathbf{z}_{-1}.$$ Therefore, $\mathbf{z}_{-1} = \lambda\mathbf{y}_{-1}$. That is, we have: \begin{align*} \mathbf{x}_1 &= \mathbf{y}_1 + \mathbf{y}_{-1},\\ \mathbf{x}_{-1} &= \mathbf{y}_1 + \lambda\mathbf{y}_{-1}. \end{align*} Finally, note that $\lambda\neq 1$, because $\mathbf{x}_1$ and $\mathbf{x}_{-1}$ are linearly independent. But this mean that $$\mathbf{W} = \mathrm{span}(\mathbf{x}_1,\mathbf{x}_{-1}) = \mathrm{span}(\mathbf{y}_1+\mathbf{y}_{-1},\mathbf{y}_1+\lambda\mathbf{y}_{-1}) = \mathrm{span}(\mathbf{y}_1,\mathbf{y}_{-1}).$$ The last equality follows since we can replace $\mathbf{y}_1+\lambda\mathbf{y}_{-1}$ with the difference between the two basis vectors, which is a nonzero multiple of $\mathbf{y}_{-1}$; then we can replace this multiple with $\mathbf{y}_{-1}$ itself, and then we can replace $\mathbf{y}_1+\mathbf{y}_{-1}$ with $\mathbf{y}_1$ by itself, since $\mathbf{y}_{-1}$ is also in the span.

Thus, there is a basis for $\mathbf{W}$, namely $[\mathbf{y}_1,\mathbf{y}_{-1}]$, consisting of eigenvectors of $B$. Therefore, $\mathbf{W}$ is $B$-invariant.

In summary, if there are no $1$-dimensional subspaces that are both $A$- and $b$-invariant, then there is a $2$-dimensional subspace that is both $A$- and $B$-invariant.

Note. Although there seems to be some lack of symmetry on $T$, since it depends on the projections on $B$, note that $(p^B_{ij})^{-1} = p^A_{ji}$, so in fact the argument is very symmetric relative to $A$ and $B$.

$\dim(\mathbf{V})=2n$, $\dim(E^A_{i})=\dim(E^B_{j})=n$, $E^A_i\cap E^B_{j}=\{\mathbf{0}\}$

and that $\mathbf{W}$ is a $2$-dimensional subspace of $\mathbf{V}$ that is $A$-invariant, and $B$-invariant.

Remember that the restriction of a diagonalizable operator to an invariant subspace is diagonalizable. So $\mathbf{W}$ must have a basis of eigenvectors of $A$ and of $B$.

If $\mathbf{W}\subseteq E^A_i$, then the eigenvectors of $B$ contained in $\mathbf{W}$ would be common eigenvectors of $A$ and $B$, which we are ruling out. So $\mathbf{W}$ cannot be contained in $E^A_i$; symmetrically, it cannot be contained in $E^B_j$. Since the restrictions of both $A$ and $B$ are diagonalizable, we can find a basis for $\mathbf{W}$ consisting of eigenvectors of $A$, and one consisting of eigenvectors of $B$. But that means that there is a basis $\mathbf{x}_1,\mathbf{x}_{-1}$ of $\mathbf{W}$ with $\mathbf{x}_i\in E^A_i$, and a basis $\mathbf{y}_1,\mathbf{y}_{-1}$ with $\mathbf{y}_j\in E^B_j$.

If we write $\mathbf{x}_i$ as \begin{align*} \mathbf{x}_1 &= \mathbf{z}_1+\mathbf{z}_{-1}\\ \mathbf{x}_{-1}&=\mathbf{w}_1 + \mathbf{w}_{-1} \end{align*} with $\mathbf{z}_j,\mathbf{w}_j\in E^B_j$, then since the $\mathbf{y}_j$ are a basis for $\mathbf{W}$ we must have that $\mathbf{z}_j,\mathbf{w}_j\in\mathrm{span}(\mathbf{y}_j)$, so the $E^B_j$ components of $\mathbf{x}_{1}$ and $\mathbf{x}_{-1}$ are multiples of the same vector; that is, they are collinear. So $p^B_{1,1}(\mathbf{x}_1)$ and $p^B_{-1,1}(\mathbf{x}_{-1})$ are collinear; likewise with their images in $E^B_{-1}$.

Symmetrically, if we write the $\mathbf{y}_j$ as a sum of vectors in $E^A_i$, then the $E^A_1$ components must both be multiples of $\mathbf{x}_1$, and the $E^A_{-1}$ components must both be multiples of $\mathbf{x}_{-1}$, hence they are collinear.

In particular, using the argument given by Plop, we see that the existence of a 2-dimensional subspace of $\mathbf{V}$ which is both $A$- and $B$-invariant, then it is not contained in $E^A_i$ or $E^B_j$, $i,j\in\{1,-1\}$, and the existence of an eigenvector for the operator $T$ defined above follows.

In summary, putting together the two parts, we have the following:

Theorem. Suppose $\mathbf{V}$ is a vector space (over a field of characteristic different from $2$), and $A$ and $B$ are operators such that $A^2=B^2=I$. Let $E^A_i$ be the eigenspace of $A$ corresponding to $i$, $E^B_j$ the eigenspace of $B$ corresponding to $j$, with $i,j\in\{1,-1\}$. Suppose further that $\dim(\mathbf{V})=2n$, $\dim(E^A_i)=\dim(E^B_j)=n$, and $E^A_i\cap E^B_j = \{\mathbf{0}\}$. Then $\mathbf{V}$ has a $2$-dimensional subspace that is both $A$-invariant and $B$-invariant if and only if the operator $$T = \left(p^B_{1,-1}\right)^{-1}\circ p^B_{-1,-1}\circ\left(p^B_{-1,1}\right)^{-1}\circ p^B_{1,1} = p^A_{-1,1}\circ p^B_{-1,-1}\circ p^A_{1,-1}\circ p^B_{1,1}$$ on $E^A_1$ has an eigenvector, where $p^A_{j,i}$ is the restriction of the projection $p^A_i\colon \mathbf{V}\to E^A_i$ along $E^A_{-i}$ to $E^B_j$, and $p^B_{i,j}$ is the restriction of the projection $p^B_j\colon \mathbf{V}\to E^B_{j}$ along $E^B_{-j}$ to $E^A_i$.

The reason we need the characteristic to be different from $2$ is so that we can use the fact that $A$ and $B$ are both diagonalizable; if the characteristic is $2$, then we don't know that, because the minimal polynomial could be $(t+1)^2$.

-
One idea given to me was to take $x\in E_1^A$ and to consider the vector space generated by $a$ and $Ba$. – Jon Beardsley Apr 6 '11 at 19:00
But thankyou, I am going to read this carefully, this is very clear. – Jon Beardsley Apr 6 '11 at 19:02
@JBeardz: Note that the example I gave with no $1$-dimensional invariant subspace for both also works if you think of it as an operator on $\mathbb{C}^2$ instead of $\mathbb{R}^2$. – Arturo Magidin Apr 6 '11 at 19:14
Thanks, I really appreciate it. – Jon Beardsley Apr 10 '11 at 14:26
@JBeardz: Having slept on it, there are a few place where it can be greatly streamlined and clarified, so I'm going to do that. – Arturo Magidin Apr 10 '11 at 18:20

Consider the linear transformation $AB:V\to V$. Since $V$ is a complex vector space, $AB$ has at least an eigenvector, call it $X$, i.e. $ABx=\lambda x$. Note, $AABx=Bx=A\lambda x=\lambda Ax$ and then $x=BBx=BA\lambda x=\lambda BAx$ So consider the space $\langle x, Ax\rangle$. Note, this space is invariant under $A$, clearly, $Bx=\lambda Ax$ and $BAx=x$ so it's invariant under $B$. This space is at least one dimensional.

-
@JBearz: Yes, that seems to work; excuse me while I go kick myself. – Arturo Magidin Apr 12 '11 at 1:57
@Jbearz: On the other hand, I don't think that my solution uses that $A^2=B^2=I$, other than to deduce that operators are diagonalizable with at most two eigenvalues. Haven't thought it through, but it's possible that the argument given will go through if all you know is that the minimal polynomial of each of $A$ and $B$ divides a squarefree quadratic (and they could be different quadratics)... – Arturo Magidin Apr 12 '11 at 2:01
@Arturo: Haha, no worries. I felt the same way. At least you were able to brute force it and get a result! I appreciate your continued involvement in this not particularly interesting problem. – Jon Beardsley Apr 12 '11 at 2:02
@Arturo: I see, so a stronger statement then, applicable to any diagonalizable linear operators? – Jon Beardsley Apr 12 '11 at 2:03
Not "any": there was a strong point on the fact that each of $A$ and $B$ had at most two eigenvalues; if there were more eigenvalues, you wouldn't be able to reduce to the case where $V$ decomposes as a direct sum of two subspaces, each of half the dimension, in two different ways that are "disjoint". But I think that if we replace $1$ and $-1$ for $A$ with $\lambda_1$ and $\lambda_2$, and replace $1$ and $-1$ for $B$ with $\mu_1$ and $\mu_2$, the constructive argument will go through regardless (and/or the necessary/sufficient condition using the projections). – Arturo Magidin Apr 12 '11 at 2:05

Arturo's answer can be condensed to the following:

Let $U_1$, $\ldots$, $U_4$ be the eigenspaces of $A$ and $B$. Letting the simple cases aside we may assume that $U_i\oplus U_j=V$ for all $i\ne j$. We have to produce four nonzero vectors $x_i\in U_i$ that lie in a two-dimensional plane.

For $i\ne j$ denote by $P_{ij}:V\to V$ the projection along $U_i$ onto $U_j$; whence $P_{ij}+P_{ji}=I$. The map $T:=P_{41}\circ P_{32}$ maps $V$ to $U_1$, so it leaves $U_1$ invariant. It follows that $T$ has an eigenvector $x_1\in U_1$ with an eigenvalue $\lambda\in{\mathbb C}$. Now put \eqalign{x_2&:=P_{32}x_1\in U_2\ ,\cr x_3&:=x_1-x_2=P_{23} x_1\in U_3\ ,\cr x_4&:=x_2-\lambda x_1=x_2- P_{41}P_{32}x_1=x_2-P_{41}x_2=P_{14}x_2\in U_4\ .\cr} It is easily checked that all four $x_i$ are nonzero.

-
What is the point of producing four such vectors, how does this show the existence of a two dimensional invariant subspace? – Jon Beardsley Apr 11 '11 at 14:48
@JBeardz: The two vectors $x_1$ and $x_2$ together span the same two-dimensional subspace $U$ as do $x_3$ and $x_4$ together. By construction $x_1$ and $x_2$ are eigenvectors of $A$, $x_3$ and $x_4$ are eigenvectors of $B$; so $U$ is invariant under $A$ as well as under $B$. – Christian Blatter Apr 11 '11 at 20:29

This is not true in general, I guess you omitted part of the question (something like "$V$ is a complex vector space").

The dimension of subspaces stable under linear operators is something that depends much on the base field $K$.

Let us assume we are in a situation where your argument (more precisely the variation thereof pointed out by Qiaochu and Arturo) does not work, i.e. $E^A_{i} \cap E^B_{j} = \{ 0 \}$ for all $i,j$. As Arturo explained, there is no one dimensional subspace invariant under $A$ and $B$ in this case. Moreover, any subspace invariant under $A$ and $B$ is not contained in any of the $E^A_i$, $E^B_j$.

The linear maps $p_{i,j} : E^A_i \rightarrow E^B_j$ which are the restriction to $E^A_i$ of the projection along $E^B_{-j}$ onto $E^B_{-j}$, are isomorphisms. A two-dimensional subspace of $V$ invariant under $A$ but not contained in $E^A_{\pm 1}$ is of the form $Kx \oplus Ky$ where $x \in E^A_1$, $y \in E^A_{-1}$ are not zero. The same is true for $B$ instead of $A$, so $p_{1,1}(x)$ and $p_{-1,1}(y)$ are colinear. We can scale $y$ and assume that $y = p_{-1,1}^{-1} \circ p_{1,1}(x)$. Similarly, $p_{1,-1}(x)$ and $p_{-1,-1}(y)$ have to be colinear, so $p_{1,-1}^{-1} \circ p_{-1,-1} \circ p_{-1,1}^{-1} \circ p_{1,1} (x) = \lambda x$, and so such an $x$ exists iff $p_{1,-1}^{-1} \circ p_{-1,-1} \circ p_{-1,1}^{-1} \circ p_{1,1}$ has an eigenvector.

Over $\mathbb{R}$, it can happen that this invertible operator has no eigenvector, for example if we take $A=\begin{pmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & -1 & 0 \\ 0 & 0 & 0 & -1 \end{pmatrix}$ and $B=\begin{pmatrix} 0 & -1 & -1 & 1 \\ 1 & 0 & -1 & -1 \\ -1 & -1 & 0 & 1 \\ 1 & -1 & -1 & 0 \end{pmatrix}$, the aforementioned operator has matrix $\begin{pmatrix} 0 & 1 \\ -1 & 0 \end{pmatrix}$ which has no real eigenvalue.

-
Oh wow! Yeah, it is a complex vector space, sorry! – Jon Beardsley Apr 6 '11 at 19:03
In that case, my previous answer gives a solution: every endomorphism of a complex vector space has an eigenvector. – Plop Apr 6 '11 at 19:12
Okay, thankyou. I'm not entirely clear on how to think of these projections on to something with respect to something else. So I will have to suss all that out. Or even why it is the case that such a vector space must be the sum of two such spaces. – Jon Beardsley Apr 6 '11 at 19:27
Sorry about that, English is not my native language. According to Wikipedia, I should have written "projection along $E^B_{-j}$ onto $E^B_j$" – Plop Apr 6 '11 at 19:34
Is it obvious that such a projection always exists, and in this case is an isomorphism? – Jon Beardsley Apr 6 '11 at 20:14