# Invariant subspaces to the permuting action of the alternating group.

This question arose while researching Galois groups of random reciprocal polynomials. Throughout $$A_n$$ is the alternating group.

Let us define the following group action of $$A_n$$ on the vector space $$\mathbb{C}^n$$

$$\sigma \circlearrowright (x_1,\dots,x_n)=(x_{\sigma^{-1}(1)},\dots,x_{\sigma^{-1}(n)})$$

My interest is to classify all the subspaces invariant to this action. My idea was to classify them in terms of linear systems. the invariant spaces I found are the following ones:

1. $$\{0\}\subseteq \mathbb{C}^n$$ which corresponds to $$x_1 = \dots = x_n = 0$$,
2. $$V_1=\operatorname{span}\{(1,\dots, 1)\}\subseteq \mathbb{C}^n$$ which corresponds to $$x_1 = \dots = x_n$$,
3. $$V_2\subseteq \mathbb{C}^n$$ which corresponds to $$\sum_{i=1}^nx_i=0$$,
4. $$\mathbb{C}^n \space$$, the whole space.

In case we extend the action to the whole symmetric group $$S_n$$ it can be shown that those truly are all the invariant subspaces, however, for $$A_n$$ I suspect I missed a few.

Any Help would be much appreciated!

• Just a minor point: that’s not a group action. The group action you’re looking for is (I think) the one that sends the $i$-th standard basis vector $e_i$ to $e_{\sigma(i)}$, so it does $(x_1,..,x_n) \mapsto (x_{\sigma^{-1}(1),..,x_{\sigma^{-1}(n)})$. Aug 28 '20 at 18:22
• Yes, you are indeed right! Thank you Aug 28 '20 at 18:35
• The invariant subspaces remain the same, so I am invariant to the choice of right or left action. Aug 28 '20 at 18:38
• The standard representation stays irreducible after restriction to the alternating group, so that your list also contains all $A_n$ invariant subspaces. I don't add this comment as an answer as I do not know an easy argument to prove it. Aug 28 '20 at 18:46
• @juandiegorojas Only if $n > 3$, $A_3$ is Abelian so doesn't have a 2 dimensional irreducible. Mark sketches a proof of irreducibility in his answer (follows from the permutation action being 2-transitive when $n > 3$). Aug 28 '20 at 18:54

If $$n=3$$ then there might be more subspaces, for example the span of $$(1,\omega,\omega^2)$$ where $$\omega=-\frac{1}{2}+i\frac{\sqrt{3}}{2}$$.

If $$n>3$$ then the subspaces you listed are indeed all the subspaces. Let $$e_1,...,e_n$$ be the standard basis vectors of $$\mathbb{C^n}$$. We can define an action of $$A_n$$ on $$\{e_1,...,e_n\}$$ by $$\sigma.e_i=e_{\sigma(i)}$$. Just like any other action, it induces a representation of $$A_n$$. Formally, the representation is defined as $$\pi:A_n\to GL(\mathbb{C^n})$$ where $$\pi(\sigma)(a_1e_1+...+a_ne_n)=a_1e_{\sigma(1)}+...+a_ne_{\sigma(n)}$$. So we are actually looking for the invariant subspaces of this representation.

We can obviously define the invariant subspaces $$V_1$$ and $$V_2$$ like you did in the question. The subspace $$V_1$$ is one dimensional, thus irreducible. Also, since $$n>3$$ the action of $$A_n$$ on $$\{e_1,...,e_n\}$$ is $$2$$-transitive. (can you prove it?). This implies that if $$\chi$$ is the character of $$\pi$$ and $$\chi_1$$ is the character of the trivial representation then $$\chi-\chi_1$$ is an irreducible character. Since $$\mathbb{C^n}=V_1\oplus V_2$$ it follows that $$\chi-\chi_1$$ is exactly the character of $$V_2$$, so $$V_2$$ is also irreducible.

Now the result follows. Take a non-trivial invariant subspace $$U$$. It has an invariant complement $$W$$, which also has to be non-trivial. By Maschke's theorem you can decompose both into a direct sum of irreducible subspaces. But such a decomposition of $$\mathbb{C^n}$$ is unique, it is exactly $$V_1\oplus V_2$$. So we have no choice, $$U$$ must be either $$V_1$$ or $$V_2$$.

• Very nice answer. I wanted to add: In the $n = 3$ case its clear from rep theory that the standard rep can't be irreducible because $A_3$ is Abelian. Also, there is a nice proof of the rep theory fact about 2-transitive actions here: mathoverflow.net/a/17244/41873 Aug 28 '20 at 18:56
• Beautiful! Thank you! And thank you, Lorenzo, for the nice proof of the number of irreducible representations of 2-transitive groups. @LorenzoNajt Aug 28 '20 at 18:57
• For completeness: when $n = 3$, the three irreducible subspaces are the one Mark mentioned, its "complex conjugate", and $V_1$. There are exactly eight invariant subspaces, namely the direct sum of any subset of the irreducibles. And of course every subspace is invariant when $n \leq 2$. Aug 28 '20 at 19:02

Suppose first that $$n \geq 4$$. You’re looking at the so-called permutation representation with respect to the standard action of $$A_n$$ on $$n$$ letters. Recall that such a representation always decomposes as a direct sum of the forms (2) and (3) (where by (2) and (3) I mean the invariant subspaces 2. and 3. you already give in your list). (3) is irreducible if and only if the action of the group is $$2$$-transitive (see theorem 7.2.11 in http://users.metu.edu.tr/sozkap/513-2013/Steinberg.pdf#page85). But I’m sure you can prove the action of $$A_n$$ is $$2$$-transtive for $$n \geq 4$$. So $$\mathbb{C}^n$$ decomposes as a direct sum of irreducible represenations $$\mathbb{C}^n=V_1 \oplus V_2$$ Let $$W \subset \mathbb{C}^n$$ be an irreducible invariant subspace. Then by unicity of decomposition of irreducible represenations (in its strongest form), $$W$$ is equal to either $$V_1$$ or $$V_2$$, and we’re done. The cases $$n=1$$ and $$n=2$$ are trivial. If $$n=3$$, $$\mathbb{C}^3$$ decomposes as $$\text{span}\{(1,1,1)\} \oplus \text{span}\{1, \zeta, \zeta^2\} \oplus \text{span}\{1,\zeta^2, \zeta\}$$ where $$\zeta$$ is a primitive third root of unity. One easily sees the representations occuring in the above decomposition are irreducible and inequivalent, so now any invariant subspace is the direct sum of any of the $$2^3=8$$ choices of subsets of $$\{ \text{span}\{(1,1,1)\}, \text{span}\{1, \zeta, \zeta^2\} , \text{span}\{1,\zeta^2, \zeta\} \}.$$

• Thank you very much! The reference you attached is very promising. I should dive into representation theory :) Aug 29 '20 at 9:05
• Honestly, I don’t really like that reference. It assumes you are afraid of tensor products and thus defines the induced representation in a really awkward ad hoc way. Also, I found that some proofs are too involved and have better proofs, by better I mean they have proofs that generalize. For example, unicity of decomposition of a representation up to iso is proven by means of characters while this follows more easily from schur’s lemma, and the schur lemma proof can be used to prove the stronger statement I needed above. Nevertheless, it’s an ok source to learn some very introductory facts. Aug 29 '20 at 18:33