# False proof that $\langle\chi,1_G\rangle$ need not be an integer.

I'd like to know where the following calculation has gone wrong. I'm sure it is a silly error.

Let $$G$$ be a finite group acting on the right cosets $$G/H$$ of $$H\le G$$. Let $$\chi$$ be the character of the permutation representation of $$G$$ defined by this action. Let $$1_G$$ be the trivial character of $$G$$.

Then for $$g\in G$$ we have that $$\chi(g)$$ is the number of points in $$H/G$$ fixed by $$g$$.

But $$g$$ fixes $$Hx$$ if and only if $$G\in H^x$$ so $$\chi(g)$$ is the number of conjugates $$K$$ of $$H$$ with $$g\in K$$.

Denoting by $$Cl(H)$$ the set of subgroups of $$G$$ conjugate to $$H$$ and $$(x,K)=\cases{1 & x\in K \\ 0 & x\notin K}$$ we have
$$\begin{array}{ll} \langle \chi, 1_G\rangle & =\frac{1}{|G|}\sum\limits_{g\in G}\chi(g) \\ & =\frac{1}{|G|}\sum\limits_{g\in G}\sum\limits_{K\in Cl(H)}(g,K) \\ & =\frac{1}{|G|}\sum\limits_{K\in Cl(H)}\sum\limits_{g\in G}(g,K) \\ & =\frac{1}{|G|}\sum\limits_{K\in Cl(H)}|K| \\ & =\frac{1}{|G|}|Cl(H)||H| \\ & =\frac{|H|}{|N_G(H)|} \\ \end{array}$$

This is clearly not always an integer, when $$\langle \chi, 1_G\rangle$$ is. Where is the error?

• It is the normaliser, but it's the other way round - $H$ is a subgroup of $N_G(H)$ so the value is not an integer if $H$ is a proper subgroup of $N_G(H)$ – Robert Chamberlain Feb 3 at 9:34
• I think the error is that you only count conjugates of $H$, whereas the character cares about which element does the conjuating ($H^x = H^y$ does not imply that $Hx = Hy$, only that the cosets for the normalizer are equal, which explains why you get this in the denominator rather than $|H|$ as you should). – Tobias Kildetoft Feb 3 at 10:00
• Thank you @Tobius, write this as an answer and I'll mark it as correct – Robert Chamberlain Feb 3 at 10:02

## 1 Answer

Your computation looks correct to me, in that $$\chi(g)=|\{x\in H\backslash G|xgx^{-1}\in H\}$$.

This leads one to \begin{equation}\begin{aligned} \langle\chi,1\rangle&=\frac1{|G|}\sum_{g\in G}\sum_{x\in H\backslash G}1_{xgx^{-1}\in H}\\ &=\frac1{|G|}\sum_{x\in H\backslash G}|xHx^{-1}|\\ &=\frac1{|G|}\sum_{x\in H\backslash G}|H|\\ &=\frac1{|G|}|H||G|/|H|=1. \end{aligned}\end{equation}

This what we would expect , since writing $$e_{Hx}$$ for the unit basis vector generated by the right coset $$Hx$$ we would get an invariant subspace spanned by $$\sum_{x\in H\backslash G}e_{Hx}$$.

What does wrong in your proof is taking the sum over Cl$$(H)$$, which has size $$|G|/|N_G(H)|$$ as opposed to $$|G|/|H|$$. This is because if you have $$xH\neq x'H$$, but $$xHx^{-1}=x'Hx'^{-1}$$, then you want to make sure to count that twice.

• Well, it is indeed what we would expect, but the reasoning here would only give an inequality. The reason we know we should get the equality is Frobenius reciprocity (essentially). But this does not really address the question of where the error is in the calculation. – Tobias Kildetoft Feb 3 at 9:51
• Thanks both, and thank you for the full calculation – Robert Chamberlain Feb 3 at 10:03