A simpler answer than my previous one:
Consider
$$H=\bigcap_{g\in G} \langle g\rangle$$
The intersection of cyclic groups is cyclic, so either
(1) $H=\langle h\rangle$ with $\text{ord}_G(h)=n<\infty$
(2) $H=\langle h\rangle\cong \Bbb Z$ whence $\langle h^2\rangle$ is a proper subgroup, contradicting that $H$ was the intersection of all cyclic subgroups of $G$.
So the intersection has finite order. Since $n\ne 1$ by assumption, every $g\in G$ has that $\langle h\rangle \le \langle g\rangle$, but every non-identity element of an infinite cyclic group has infinite order, and clearly $h\in\langle g\rangle$ does not, so $|\langle g\rangle|<\infty$