Prove that $H=\{x\in G \mid axb=bxa\}$ is a subgroup of $G$.

The question is. Let $G$ be a group and let $a,b\in G$ be two fixed elements which happen to commute $(ab=ba)$. Prove that $H=\{x\in G \mid axb=bxa\}$ is a subgroup of $G$.

The book goes over the proof: $axb=bxa$ implies that $x(ba^{-1})=(a^{-1}b)x$. And since $ab=ba$, it follows that $ba^{-1}=a^{-1}b$. So, $H$ is a subgroup.

I was wondering if someone could walk me through a more formal proof for a subgroup with this (i.e showing it commutes, holds for inverses, etc.).

• Just because this doesn't do the usual tests directly does not mean it's not a "formal" proof. This is just as rigorous, and arguably better if there's no obvious way to check the closure properties without symbolic manipulations that mimic this proof anyway. (Also, how is "showing it commutes" part of showing a subset is a subgroup, and what does "it" in this phrase refer to anyway?) – anon Dec 16 '13 at 17:18
• (Also, as Ian notes, this proof is not quite finished; one should for example remark that $H$ is the set of fixed points of a particular conjugation automorphism, and either argue or invoke the fact that fixed points of an automorphism pass all the subgroup test.) – anon Dec 16 '13 at 17:24

Claim: $e\in H$. Well since $ab=ba$, $aeb=ab=ba=bea$, so that's fine.
Claim: for $x,y\in H$, $xy\in H$. Well $axyb=(axb)b^{-1}a^{-1}(ayb)=(bxa)b^{-1}a^{-1}(bya)$. Now since $a,b$ commute, their inverses do too. So we hae $axyb=(bxa)a^{-1}b^{-1}(bya)=bxya$, as required.
Other strategy: $H$ is precisely the centraliser of the element $b^{-1}a$. Centralisers are subgroups. Therefore $H$ is a subgroup.