If $G\neq 1$, then $G$ being locally graded, it has a proper subgroup of finite index $H$. Hence $H$ is abelian; in particular $G$ is virtually abelian.
Assume that $G$ is not abelian and infinite. Then any two non-commuting elements of $G$ generate $G$; in particular $G$ is finitely generated.
Being finitely generated and virtually abelian, $G$ has a normal subgroup of finite index $N$ that is free abelian of finite rank. Pick a prime $p$ greater than $|G/N|$, and let $q>p$ be another larger prime. Then the finite group $H=G/N^{pq}$ also has all its proper subgroups abelian; it's an extension of $N/N^{pq}$ by $G/N$, which have coprime order; hence this extension is split, so $H=(N/N^{pq})\rtimes K$ with $K\simeq G/N$. Since $N/N^{pq}$ splits as the direct product of its Sylows $N/N^p$ and $N/N^q$, $H$ has the proper subgroup $N/N^q\rtimes K$. Hence $N/N^q\rtimes K\simeq G/N^q$ is abelian. Since this holds for every prime $q>p$ and $\bigcap_{q>p}N^q=1$, we deduce that $G$ is abelian, contradiction.