# The set of all $x$ such that $xHx^{-1}\subseteq H$ is a subgroup, when $H\leq G$

I found this problem in a textbook of abstract algebra:

Let $H$ be a subgroup of $G$. Prove that $$\{x\in G:xax^{-1}\in H\text{ for every }a\in H\}$$ is a subgroup of $G$.

It's easy to prove that the set is closed under multiplication, but I'm stuck on proving that it is closed under inverses.

If $H$ is finite, say $H=\{a_1,\ldots,a_n\}$, suppose $x$ is an element of the set. Then $xa_1x^{-1},\ldots,xa_nx^{-1}$ are all distinct, hence they are exactly $a_1,\ldots,a_n$, in some order. Therefore any element $b\in H$ can be written as $xcx^{-1}$ for some $c\in H$, and hence $x^{-1}bx=x^{-1}(xcx^{-1})x=c$ is also in $H$. So $x^{-1}$ is also an element of the set.

However, the above method does not work if $H$ is infinite. The main idea is to prove that $x^{-1}ax\in H$ for every $a\in H$, given that $xax^{-1}\in H$ for every $a\in H$. I was trying to do some substitutions of $a$ to get the required result, but I can't seem to get the $x^{-1}$ to the left.

Any help would be appreciated. It may be worth mentioning that I just started learning this group theory thing for a few days, so please adjust your explanation accordingly.

-

The reason you are having trouble proving it is that it is not true as stated.

For a heavy-handed example, let $G$ be the free group on $x$ and $y$, and let $H$ be the subgroup generated by all elements of the form $x^nyx^{-n}$ with $n\gt 0$.

Then for any $a\in H$ we have $xax^{-1}\in H$. However, $x^{-1}yx\notin H$, because any element of $H$ is a word that starts with a nonnegative power of $x$.

To fix the problem, you would need to require $xHx^{-1}=H$, rather than $xHx^{-1}\subseteq H$. Then your argument would go through in the infinite case as well.

Okay, here's an example you can get your hands on (courtesy my Math 257 notes with T.Y. Lam, Spring 97):

Let $G$ be the group of all invertible $2\times 2$ matrices with coefficients in $\mathbb{Q}$, $G=\mathrm{GL}_2(\mathbb{Q})$.

Let $H$ be the subgroup given by $$H = \left\{\left.\left(\begin{array}{cc} 1 & m\\ 0 & 1\end{array}\right)\in G\ \right|\ m\in\mathbb{Z}\right\}.$$ Let $$x = \left(\begin{array}{cc} 2 & 0\\ 0 & 1\end{array}\right), \qquad x^{-1} = \left(\begin{array}{cc} \textstyle\frac{1}{2} & 0 \\ 0 & 1 \end{array}\right).$$ Then for every $m\in\mathbb{Z}$ we have: \begin{align*} \left(\begin{array}{cc} 2 & 0\\ 0 & 1\end{array}\right)\left(\begin{array}{cc}1 & m\\0 & 1\end{array}\right) \left(\begin{array}{cc}\textstyle\frac{1}{2}&0\\0&1\end{array}\right) &= \left(\begin{array}{cc} 2 & 2m\\0 & 1\end{array}\right)\left(\begin{array}{cc} \textstyle\frac{1}{2}&0\\0 & 1\end{array}\right)\\ &= \left(\begin{array}{cc} 1 & 2m\\ 0 & 1\end{array}\right)\in H. \end{align*} So $xHx^{-1}\subseteq H$. However, even though $$\left(\begin{array}{cc}1&1\\0&1\end{array}\right)\in H,$$ we have \begin{align*} x^{-1}\left(\begin{array}{cc}1&1\\0&1\end{array}\right)x &= \left(\begin{array}{cc} \textstyle\frac{1}{2} & 0\\0 &1\end{array}\right)\left(\begin{array}{cc}1 & 1\\ 0 & 1\end{array}\right)\left(\begin{array}{cc}2 & 0\\0&1\end{array}\right)\\ &= \left(\begin{array}{cc} \textstyle\frac{1}{2}&\textstyle\frac{1}{2}\\0&1\end{array}\right)\left(\begin{array}{cc}2&0\\0&1 \end{array}\right)\\ &= \left(\begin{array}{cc} 1 & \textstyle\frac{1}{2}\\0&1\end{array}\right)\notin H. \end{align*} So $x\in \{g\in G\mid ghg^{-1}\in H\text{ for all }h\in H\}$, but $x^{-1}\notin\{g\in G\mid ghg^{-1}\in H\text{ for all }h\in H\}$. So the set need not be closed under inverses.

-
It's a relief to know that the question is wrong, because the other questions in the set are merely computations and short proof exercises. I suspected that something was weird here, but didn't go as far to find a counterexample because I thought that the textbook couldn't be wrong. –  fred Oct 25 '11 at 3:34
@fred: Which textbook is it? –  Arturo Magidin Oct 25 '11 at 3:39
@fred Looking at my copy of Pinter you appear to have mis-copied the problem. The original statement is show that $K=\{x \in G : xax^{-1} \in H \text{ iff } a \in H\}$ if a subgroup of $G$. –  JSchlather Oct 25 '11 at 4:26
@Jacob: That would indeed make the problem true, since that would correspond to $xHx^{-1}=H$, instead of $xHx^{-1}\subseteq H$. –  Arturo Magidin Oct 25 '11 at 4:45
In my copy, it's written "for every" instead of "iff". Probably the author correct it in the newer edition. –  fred Oct 25 '11 at 5:12