# Restriction of irreducible representation to subgroup of index 2

Let $$G$$ be a finite group, $$H \unlhd G$$, $$[G:H] = 2$$, $$\pi$$ - irreducible complex representation of $$G$$, $$\epsilon$$ is also a representation of $$G$$ such that $$\epsilon(g) = id_{\mathbb{C}}$$ if $$g \in H$$ and $$\epsilon(g) = -id_{\mathbb{C}}$$ if $$g \not \in H$$.

Suppose $$\pi \cong \pi \otimes \epsilon$$. I need to prove the following:

1. $$Res_H^G\pi = \phi_1 \oplus \phi_2$$, where $$\phi_1$$ and $$\phi_2$$ are both irreducible representations of $$H$$
2. $$\phi^g_1 \cong \phi_2$$, where $$\phi^g_1(h) = \phi_1(ghg^{-1})$$ for some $$g \in G\backslash H$$

I managed to prove the first part of the question using some character theory (proving that the restriction can either be irreducible or a direct sum of two irreducible representations), but I can't do the second one.

I tried proving that $$\phi'_1 \not \cong \phi_1$$ to start with, but couldn't even do that. Any help will be appreciated.

Thanks!

• I think you mean you start with proving $\phi_1\not\cong\phi_2$. If $\phi_1\cong\phi_2$ then $\langle\operatorname{Res}^G_H\pi,\operatorname{Res}^G_H\pi\rangle_H=4$ not $2$. Commented Oct 9, 2023 at 20:00

Both induction and restriction between $$H$$ and $$G$$ can be very precisely described because the index is two. This is almost certainly overkill for this particular question, but I find it clarifies things.

In what follows, let $$V$$ be a representation of $$G$$, and $$W$$ a representation of $$H$$. Let $$\omega \in G \setminus H$$. We write $$W^\omega$$ to denote the representation of $$H$$ obtained from $$W$$ by first conjugating $$H$$ by $$\omega$$.

From the formula of the character of an induced representation, one has the following two identities: $$\text{Res}_H^G \text{Ind}_H^G W = W \oplus W^\omega, \\ \text{Ind}_H^G \text{Res}_H^G V = V \oplus (V \otimes \epsilon).$$ Coupling this together with Frobenius Reciprocity $$(\text{Res}_H^G V, W)_H = (V, \text{Ind}_H^G W)_G$$ we have that $$(\text{Res}_H^G V, \text{Res}_H^G V)_H = (V,V)_G + (V, V \otimes \epsilon)_G,\\ (\text{Ind}_H^G W, \text{Ind}_H^G W)_G = (W,W)_H + (W, W^\omega)_H$$

Therefore, when $$V$$ and $$W$$ are irreducible, the induction / restriction is either irreducible, or the sum of two non-isomorphic irreducible representations.

Using these identities, we can show the following claim.

Claim: Suppose that $$V$$ is an irreducible representation of $$G$$ and $$W$$ is an irreducible subrepresentation of $$\text{Res}_H^G V$$. Then there are exactly two cases that can occur:

Case 1: $$V \otimes \epsilon \cong V$$, $$W \not\cong W^\omega$$, $$\text{Ind}_H^G W = V$$, $$\text{Res}_H^G V = W \oplus W^\omega$$.

Case 2: $$V \otimes \epsilon \not\cong V$$, $$W \cong W^\omega$$, $$\text{Ind}_H^G W = V \oplus (V \otimes \epsilon)$$, $$\text{Res}_H^G V = W$$.

Proof: There are at most 4 cases, depending on whether $$W \cong W^\omega$$ and whether $$V \cong V \otimes \epsilon$$. We first show that the two pairs above are the only two options of the four.

Suppose first for a contradiction that $$V \otimes \epsilon \cong V$$ and $$W \cong W^\omega$$. Then $$\text{Res}_H^G V = W \oplus U$$ is the direct sum of two irreducible non-isomorphic representations. Then $$\text{Ind}_H^G \text{Res}_H^G V = \text{Ind}_H^G W \oplus \text{Ind}_H^G U$$ and $$\text{Ind}_H^G W$$ is the direct sum of two non-isomorphic sub-representations. In particular, $$\text{Ind}_H^G \text{Res}_H^G V$$ has length $$\geq 3$$. However $$\text{Ind}_H^G \text{Res}_H^G V = V \oplus (V \otimes \epsilon)$$ has length 2, this is a contradiction.

Suppose next for a contradiction that $$V \otimes \epsilon \not\cong V$$ and $$W \not\cong W^\omega$$. Then both $$\text{Ind}_H^G W$$ and $$\text{Res}_H^G V$$ are irreducible. However this means that $$\text{Res}^G_H V = W$$, and so $$\text{Ind}_H^G \text{Res}_H^G V = \text{Ind}_H^G W$$ has length $$1$$, which again is a contradiction.

Therefore, we are left with analysing the two cases above.

In Case 1, $$\text{Ind}^G_H W$$ is irreducible, and by the F.R. formula above, $$V = \text{Ind}^G_H W$$. Furthermore, again by this formula, $$(W, \text{Res}^G_H V) = 1$$, and similarly $$(W^\omega, \text{Res}^G_H V) = 1$$ because $$\text{Ind}^G_H W = \text{Ind}^G_H W^\omega$$.

In Case 2, $$\text{Res}_H^G V$$ is irreducible and thus $$\text{Res}_H^G V = W$$. By F.R. again, this means that $$(V, \text{Ind}^G_H W) = 1$$, and similarly again by F.R. $$(V \otimes \epsilon, \text{Ind}^G_H W) = 1$$ because $$\text{Res}_H^G V = \text{Res}_H^G (V \otimes \epsilon)$$.

• Thank you so much for such a detailed answer! Commented Oct 11, 2023 at 13:26