As Cam McLeman comments, Lagranges theorem is considerably simpler for groups of prime order than for general groups: it states that the group (of prime order) has no non-trivial proper subgroups.
I'll use the following
Lemma
Let $G$ be a group, $x\in G$, $a,b\in \mathbb Z$ and $a\perp b$. If $x^a = x^b$, then $x=1$.
Proof: by Bezout's lemma, some $k,\ell\in\mathbb Z$ exist, such that $ak+b\ell=1$. Then
$$ x = x^{ak+b\ell} = (x^a)^k \cdot (x^b)^\ell = 1^k \cdot 1^\ell = 1 $$
(If you know a little ring theory, you might prefer to notice that the set $\{i | x^i=1\}\subseteq \mathbb Z$ forms an ideal which must contain $(a,b)=1$ if it contains $a$ and $b$.)
The question
Now let $P$ be an arbitrary group of prime order $p$. Consider any $x\in P$ such that $x\neq 1$ and consider the set
$$ S = \{ 1, x, x^2 , \dots , x^{p-1} \}\subseteq P.$$
First assume two of these elements are equal, say $x^u=x^v$ and $u<v$ without loss of generality. Then $x^{v-u}=1$ and $1\leq v-u \leq p-1$. But then surely $v-u \perp p$. By the lemma, $x^{v-u} = x^p = 1$ now implies that $x=1$, a contradiction so every two members of $S$ must be different.
But then $|S|=p$. This implies $S=P$ and $P=\langle x\rangle$ is cyclic.