Units in a group ring $\mathbb{Z}[G]$

This is a homework question:

For a group G, let $g\in G$ have finite order, such that $\langle g\rangle$ is not a normal subgroup of $G$. Then $\mathbb Z[G]$ has a unit other than $\pm h$ with $h\in G$

My attempt at solving it goes like this.

The subgroup $\langle g\rangle = \{ e,g,\ldots ,g^k \}$ for some $k$, since $g$ has finite order. This is not a normal subgroup, so there exists some $x \in G$ and $j\in \{0,\ldots,k\}$ such that $xg^j x^{-1} \notin \langle g\rangle$. Now I need to show the existence of some $\epsilon \in \mathbb Z[G]^*$ (the unit group of $\mathbb Z[G]$) such that $\epsilon \notin G$ but I cannot do it. I tried to use the fact that for the previous $j$ and $x$ we stil have that $$e = hg^jh ^{-1} hg^{k-j}h^{-1},$$ but I honestly do not know how to continue.

Thanks in advance for any help