Let $G$ be a group and $H$ a subgroup of $G$. Show that $aH=Ha$ if and only if $aha^{-1} \in H$ for every $h \in H$. 
Let $G$ be a group and $H$ a subgroup of $G$. Show that $aH=Ha$ if and only if $aha^{-1} \in H$ for every $h \in H$.

Suppose that $aH= Ha$. Then for $h' \in aH$ we have that $h'=ah$ for some $h \in H$. Right multiplying by $a^{-1}$ we have that $h'a^{-1}=aha^{-1}$. As $aH= Ha$ we have that $h'=ah=ha$ so $h'a^{-1} = \underbrace{haa^{-1}}_{h} = aha^{-1} \implies aha^{-1} \in H$.
Conversely suppose that $aha^{-1} \in H$. The claim is that $aH=Ha$ so we will do both inclusions. Let $h' \in aH$. Then $h'=ah$ for some $h \in H$. Now right multiplying by $a^{-1}$ we have that $h'a^{-1}=aha^{-1} \in H$. So multiplying again from right I have that $h'a^{-1}a=aha^{-1}a \implies h'=ah \in Ha$.
For the other direction let $h'' \in Ha$. Then $h''=ha$ for some $h \in H$. Right multiplying by $a^{-1}$ then left multiplying by $a$ and again right multiplying by $a^{-1}$ I get that $ah''a^{-1}a^{-1} = aha^{-1} \in H$. Now left multiplying by $a$ I have that $aah''a^{-1}a^{-1} = aaha^{-1} \in aH$
Is it necessarily true that $aah''a^{-1}a^{-1} = h''$ and that $aaha^{-1} = ah$? This would seem to conclude the result, but I'm not sure if it's allowed.
Also is should I do these kind of "multiplying" here? It seems that I am going in circles multiplying everywhere.
 A: 
As $aH= Ha$ we have that $h'=ah=ha$...

Unfortunately this does not follow: we simply have an equality of the sets $a H = H a$. We could fix it up like this:

Suppose that $a H = H a$ and fix $h \in H$. We need to show that $a h a^{-1} \in H$. We at least know that $a h \in a H$. Since $a H = H a$
this means that $a h \in H a$ and so there exists $h' \in H$ such that
$a h = h' a$. Thus $a h a^{-1} = h'$, so in particular we can conclude
$a h a^{-1} \in H$, as we desire.

Your argument for the containment $a H \subset H a$ seems good to me. As explained in comments, there is a problem with the other containment, which indeed does not hold unless in the statement we are qualifying over all $a$ in $G$ at once (and kabenyuk gives a counterexample in this case in another answer).
A: There is an error at the beginning, when you write $h'=ah=ha$, as this implies that $a$ commutes with $h$. Further, it seems overly complicated.
Here's a hint: the hypothesis  $aH=Ha$ means that for every $h\in H$, there exists $h'\in H$ such that $\;ah=h'a$, whence $aha^{-1}=h'$ indeed belongs to $H$. The converse is obvious.
A: The converse is not true.
That is, from the condition
that $aha^{-1}\in H$ for all $h\in H$ it does not follow that $aH=Ha$.
Here is an example.
Let
$$
G=
\begin{pmatrix}
\mathbb{Q} & \mathbb{Q}\\
0 & 1
\end{pmatrix},\
H=
\begin{pmatrix}
1 & \mathbb{Z}\\
0 & 1
\end{pmatrix},\
a=
\begin{pmatrix}
2 & 0\\
0 & 1
\end{pmatrix}.
$$
It is not difficult to check that $aha^{-1}\in H$ for all $h\in H$, but
$$
aH=
\begin{pmatrix}
2 & 2\mathbb{Z}\\
0 & 1
\end{pmatrix}
\neq
\begin{pmatrix}
2 & \mathbb{Z}\\
0 & 1
\end{pmatrix}
=Ha.
$$
