The quaternion group is non-abelian p-group with every proper subgroup cyclic. In general, if $G$ is a non-abelian $p$-group with every proper subgroup cyclic, is it necessarily generalized quaternion?
Yes. If all proper subroups are cyclic, the group must have a unique subgroup of order $p$ (if it contains two subgroups of order $p$, one of them can be taken to lie in the center and thus their product is a non-cyclic subgroup). It is a basic result in the theory of $p$-groups that a $p$-group with a unique subgroup of order $p$ is either cyclic or generalized quaternion (see for instance Proposisition 1.3 of Berkovich's "Groups of Prime Power Order"). However, if $G$ is generalized quaternion of order $>8$ then $G/Z(G)$ is a dihedral group which contains a proper non-cyclic subgroup, and this subgroup then corresponds to a non-cyclic subgroup of $G$. So the quaternion group of order 8 really is the only non-abelian $p$-group with this property.