If $gHg^{-1} \subset H$, must we have $g^{-1}Hg \subset H$? Let $H$ be a subgroup of a group $G$. Consider the set $\{g \in G: gHg^{-1} \subset H\}$. Must this set always be a group? If $H$ was a finite subgroup then $gHg^{-1} \subset H$ if and only if $gHg^{-1} = H$ and so the answer to the question is yes. But what if $H$ is infinite?
In the case of infinite $H$, the only thing I am not sure about is if $g \in G$ such that $gHg^{-1} \subset H$, must $g^{-1}Hg \subset H$?
 A: It is not necessarily true for infinite groups. Counterexample:
Let $G$ be the group of permutations of $\mathbb Z$.
Let $H$ be the subgroup that fixes all of the negative numbers.
Let $g$ be map $n\mapsto n+1$.

This example is continuum large. But we can also get an example of countable size by restricting $G$ (and $H$) to permutations $\sigma$ where there exists $t\in\mathbb Z$ such that $\sigma(n)=n+t$ for all but finitely many $n$.
A: Let $G = {\rm GL}_2(\mathbf Q)$, the group of invertible $2 \times 2$ matrices with rational entries. Fix an integer $N > 1$. Inside $G$ is the subgroup $H$ of matrices of the form 
$$
\left(\begin{array}{cc}
N^{\mathbf Z} & {\mathbf Z}[1/N]\\0&1
\end{array}
\right) = 
\left\{
\left(\begin{array}{cc}
N^k & a/N^\ell\\0&1
\end{array}
\right) : k \in \mathbf Z, \ell \in \mathbf Z_{\geq 0}, a \in \mathbf Z 
\right\}.
$$
For any matrix $g_x = (\begin{smallmatrix}x&0\\0&1\end{smallmatrix})$ where $x \in \mathbf Q^\times$, we have 
$$
g_x\left(\begin{array}{cc}N^k&a/N^\ell\\0&1\end{array}\right)g_x^{-1} = \left(\begin{array}{cc}N^k&ax/N^\ell\\0&1\end{array}\right),
$$
which has the effect of simply multiplying the upper right entry of a matrix in $H$ by $x$. When $x$ is a nonzero integer we see that $g_xHg_x^{-1} \subset H$. But if $x$ is the reciprocal of an integer that is not a power of $N$, then $g_xHg_x^{-1} \not\subset H$ because $(\begin{smallmatrix}1&1\\0&1\end{smallmatrix})$ is in $H$ while
$g_x(\begin{smallmatrix}1&1\\0&1\end{smallmatrix})g_x^{-1} = (\begin{smallmatrix}1&x\\0&1\end{smallmatrix})$ is not. Taking $x = N+1$, for instance, $g_xHg_x^{-1} \subset H$ and $g_x^{-1}Hg_x = g_{1/x}Hg_{1/x}^{-1} \not\subset H$.
(The answer to the question Does $gHg^{-1}\subseteq H$ imply $gHg^{-1}= H$? gives this example with $N = 2$, in the form of a semidirect product.)
