I was wondering if there were a way to prove this without invoking the full force of the Sylow theorems. Here is my attempt:
Suppose $G$ is a group, a $p$ prime divides its order, and $p^2 > |G|$. By Cauchy's theorem, there exists a subgroup $H$ such that $|H| = p$. Suppose there exists another subgroup $K$, $|K| = p$, $K \neq H$. Since $H$ and $K$ are of prime order, any nontrivial element in each generates the entire subgroup, so we must have $H \cap K = \{e\}$. By counting argument, we have $$ |HK| = \frac{|H| |K|}{|H \cap K|} = p^2 > |G|,$$ contradicting $HK \subseteq G$. Therefore, such a $K$ does not exist, and we conclude $H$ is the unique subgroup of $G$ of order $p$.
This uniqueness implies $H \triangleleft G$: for all $g \in G$, $gHg^{-1}$ is a subgroup of order $p$, so we must have $gHg^{-1} = H, \forall g \in G$.
Is there anything inconsistent in this proof? If not, are there even more elementary proofs?