$p$-group of odd order with all proper subgroups cyclic

If $p$ is an odd prime and $G$ is a group of order $p^r$, with $r\ge 3$, such that all proper subgroups of $G$ are cyclic, then is $G$ cyclic ?

Yes, that is true. Apply induction on $r$. For $r=3$ the claim holds since there are three types of abelian groups and two extraspecials to check. All these contain subgroups isomorphic to $C_p \times C_p$, except of course $C_{p^3}$.
Now suppose $r>3$. The inductive hypothesis tells you that all maximal subgroups of the group must be cyclic. Perhaps there is an easier way to finish off the problem, but one way is this.
Certainly $C_{p^{r-1}} \times C_p$ contains a subgroup isomorphic to $C_p \times C_p$. As per this standard classification of $p$-groups with a cyclic maximal subgroup in the link, it suffices to show that the group
$$M_r(p) = \left\langle x, y : x^{p^{r-1}} = y^p = 1, x^y = x^{1+p^{r-2}} \right\rangle$$
contains a $C_p \times C_p$ subgroup. Take $H = \left\langle x^p,y \right\rangle \cong C_{p^{r-2}} \times C_p$ (do you see why?) and consider an appropriate subgroup of $H$.