Showing $G$ is infinite cyclic (or otherwise) Suppose we have a group $G$ which has an infinite cyclic normal subgroup $H$ such that $G/H \cong \mathbb{Z}_2$. We write $G/H$ as $\{H,gH\}$ for $g \notin H$.
Then suppose $g^2 \neq e$. So, we know $g^2 \in H$, but $g$ has infinite order.
I believe that $G \cong \mathbb{Z}$, but I haven't been able to come up with a complete argument. For example, it looks like $H$ corresponds to the even numbers, and $gH$ corresponds to the odd numbers, but I have yet to show that $G$ is abelian. If it is not necessarily the case that $G \cong \mathbb{Z}$, feel free to ignore everything from here onwards.
In particular, I wish to show that $\langle g \rangle$ commutes with $H$. Sure, I know that $\langle g^2 \rangle$ commutes with $H$ since $\langle g^2 \rangle \subseteq H$, but I cannot do the same for $\langle g \rangle$. All I'm certain of is that $gh = h'g$ for $h,h' \in H$; I have not been able to show that $h = h'$.
I've also tried using commutators ($[G,G]$ is contained in $H$), but I'm unsure if this is even helpful. For example, $ghg^{-1}h^{-1} = (h')h^{-1}$, but again I hit the same wall as above.  
Another guess is somehow using the quotient group (like $ghH = (gH)(hH) = (hH)(gH) = hgH$) but I believe that's faulty. Any ideas on how to proceed here?
Note: I preferably do not want to use results regarding finitely generated abelian groups, because this should be doable without it. No need for homomorphisms or automorphisms, either (not that I'm certain they'd help).
This question is closely related, but not quite the same; and it is unanswered at the time of posting this question.
 A: Take the infinite dihedral group, which is very similar to the group in the deleted answer: 
$$\;D_\infty=\Bbb Z\rtimes\Bbb Z_2\;\text{,  and the action}\;\;x^a=x^{-1}\;,\;\;\Bbb Z_2=\langle a\rangle\;,\;\;\Bbb Z=\langle x\rangle\;$$ 
This already gives you a counterexample to what you believe.
But that we can also write $\;D_\infty=\langle a,x\;/\;a^2=1,\,x^a=x^{-1}\rangle\;$, and forget now the semi direct product written above.
Now you prove: $\;H:=\langle x\rangle\;$ has index two and thus...
A: Let $G = \mathbb Z\times \mathbb Z_2$, $H = \mathbb Z\times \{0\}$,  and $g=(1,1)$.

Edit 10/29/16: I will respond to a comment.
Assume that $[G:H]=2$, $H$ is infinite cyclic, and $g\in G-H$. As Derek Holt has already mentioned, the action of $g$ by conjugation on $H$ is an automorphism that fixes $g^2\in H$. If $g^2\neq e$, then this automorphism must be the identity. In this case $G$ is abelian. The contrapositive statement is that: if $G$ is nonabelian, then any $g\in G-H$ satisfies $g^2=e$. In this case, choose $g\in G-H$ and let $C=\langle g\rangle$. $C$ is a complement of $H$, so $G$ is a nonabelian semidirect product of an infinite cyclic group by a 2-element group. In this case, $G$ must be the infinite dihedral group. 
Thus, if $G$ is not the infinite dihedral group, then $G$ is abelian. If $g^2\in H$ has a square root in $H$, that is, if $g^2=h^2$ for some $h\in H$, then $g_1:=gh^{-1}\in G-H$ satisfies $g_1^2=e$. Hence 
$C=\langle g_1\rangle$ is a complement of $H$, so $G$ is a product of an infinite cyclic group and a $2$-element group. In this case, $G\cong \mathbb Z\times \mathbb Z_2$.
Finally, if $G$ is neither of the two groups mentioned, then it is an abelian group, and for any $g\in G-H$ the element $g^2 (\in H)$ does not have a square root in $H$. If $H = \langle h\rangle$, then $g^2$ must be an odd power of $h$, say $g^2=h^{2r+1}$. Then $g_2:=gh^{-r}\in G-H$ and $\langle g_2\rangle$ contains $g_2^2 = h$ and $g_2^{2r+1} = g_2(h^r) = g$, so $\langle g_2\rangle = \langle h,g\rangle = G$. This proves that $G$ is cyclic in this last case. 
