# Proving the trace of a representation is equal to zero

Let ($$\rho$$, $$V$$) be a representation of $$G$$, so $$\rho$$: $$G$$ $$\to$$ $$GL(V)$$ is a group homomorphism. Let $$H\subsetG$$ be a normal subgroup of index 2. Suppose $$V$$ is $$G$$-irreducible, but not $$H$$-irreducible. Prove tr($$\rho$$($$g$$))=0 for $$g\notinH$$.

I thought about starting by saying $$G/H$$ is isomorphic to $$C_2$$, and so you could say that some character

$$\lambda$$= $$\begin{cases} 1, & \text{if g \in H} \\ -1, & \text{if g \notin H} \end{cases}$$

But I don't think this is very helpful, and I feel like this is not the way to go about the proof. Any help would be appreciated!

Let $$\chi$$ be the character of $$\rho$$. From orthogonality of characters, $$\sum_{g\in G}|\chi(g)|^2=|G|$$ since $$\rho$$ is irreducible on $$G$$, and $$\sum_{g\in H}|\chi(g)|^2=m|H|=\frac m2|G|$$ where $$m\ge2$$ is an integer, since $$\rho$$ is reducible on $$H$$. Then $$|G|\ge\frac m2 |G|$$, so $$m=2$$ and we have $$\sum_{g\notin H}|\chi(g)|^2=0$$ etc.

• Thanks for the answer! Sorry if this is a dumb question, but can you explain a tiny bit the conclusion? Why does m=2 line mean that $\sum_{g\notin H}$$\lvert$$\chi$($g$)$\rvert$^2 = 0? – empmoth Dec 9 '18 at 20:54
• Because $\sum |\chi(g)|^2$ over all $g \in G$ is $|G|$ by the orthogonality relations, so if the sum over $g \in H$ is $|G|$ then the sum over $g \not \in H$ must be 0. – Ted Dec 9 '18 at 22:55
• You're right I can't believe I didn't see that. Thanks for the help! – empmoth Dec 10 '18 at 0:21

Here is an alternative solution that does not directly use character theory. We are told that $$H$$ does not act irreducibly on $$V$$, so let $$W$$ be a nonzero subspace of $$V$$ of smallest dimension that is invariant under the action of $$H$$.

Now choose $$g \in G \setminus H$$. The normality of $$H$$ in $$G$$ implies that $$g(W)$$ is also invariant under that action of $$H$$. Also, since $$g^2 \in H$$, $$g^2(W) = W$$.

Now $$W + g(W)$$ is invariant under $$G$$ and so, since $$G$$ acts irreducibly on $$V$$, we have $$V = W + g(W)$$. Also, by minimality of $$W$$, we must have $$W \cap g(W) = \{0\}$$, so $$V = W \oplus g(W)$$.

Now, all elements of $$G \setminus H = gH$$ interchange the $$H$$-invariant subspaces $$W$$ and $$g(W)$$ so, by choosing a basis of $$V$$ that consists of a union of bases of $$W$$ and of $$g(W)$$, we see that the matrices of the elements of $$G \setminus H$$ with respect to this basis have trace $$0$$.