Let $G$ be a group and $H$ be a subgroup. Consider the following set: $$\hat{N}(H):=\left\{g\in G: g^{-1}Hg \subset H\right\} $$ The normalizer of $H$ is usually defined as $$N(H):=\left\{g\in G: g^{-1}Hg=H\right\} $$ My question is: are these two sets equal? If $H$ is finite, then the answer is certainly positive. The map $x\mapsto gxg^{-1}$ is an injective map from $H$ onto $H$, so if $H$ is finite, it is also invertible. Thus $g^{-1}Hg\subset H$ implies $g^{-1}Hg=H$, and $\hat{N}(H)=N(H)$.

In the general case, my guess would be no. The normalizer is a subgroup of $G$, whereas $\hat{N}(H)$ possibly isn't. While $1 \in \hat{N}(H)$ and $g,h \in \hat{N}(H)\Rightarrow gh \in \hat{N}(H)$, if $g\in \hat{N}(H)$ i cannot deduce that $g^{-1}\in \hat{N}(H)$. I couldn't find any counterexample, though. I know that $H$ would have to be infinite and nonabelian (otherwise $g^{-1}Hg=gHg^{-1}$). I tried to use some subgroups of $\operatorname{GL}_2(\mathbb{C})$, but all the ones I could construct had the whole $\operatorname{GL}_2(\mathbb{C})$ as their normalizer.


Here is a counterexample.

Take $G=\operatorname{Sym}(\Bbb Z)$, the group of permutations of the integers, and let $H$ be the subgroup of permutations that leave the natural numbers fixed. Let $g$ be the permutation $m\mapsto m+1$. Then $g\in \hat{N}(H)$ : indeed, if $\sigma\in H$, then for all $n\geq 0$ we have $(g^{-1}\sigma g)(n)= \sigma(n+1)-1 = n$ because $n+1\geq 0$. But $g^{-1}\notin \hat{N}(H)$, because if we take $\tau=(-1\; -2)\in H$, we get $(g\tau g^{-1})(0)=\tau(0-1)+1=-2+1=-1$, so $g\tau g^{-1}\notin H$.

| cite | improve this answer | |
  • $\begingroup$ Nice example! So as your example shows, if $G$ is infinite, $\hat{N}(H)$ need not even be a subgroup of $G$ (since it's not closed under inverses). $\endgroup$ – quasi Sep 1 '17 at 11:13
  • 1
    $\begingroup$ Actually, the issue here is precisely the fact that it is not a subgroup. Suppose that $\hat{N}(H)$ is a subgroup, and let $g\in \hat{N}(H)$. Then $g^{-1}\in \hat{N}(H) \Rightarrow gHg^{-1}\subset H \Rightarrow H\subset g^{-1}Hg$, and hence $H=g^{-1}Hg$. So $\hat{N}(H)=N(H)$ if and only if $\hat{N}(H)$ is a subgroup. $\endgroup$ – Lorenzo Quarisa Sep 1 '17 at 12:21

Not the answer you're looking for? Browse other questions tagged or ask your own question.