Problem of finding subgroup without Sylow's Thm. Let $G$ is a group with order $p^n$  where $p$ is prime and $n \geq 3$.
By Sylow's Thm, we know that $G$ has a subgroup with order $p^2$.
But, I wonder to proof without Sylow's Thm.
 A: Theorem. Let $G$ be a group of order $p^n$, where $p$ is a prime and $n\gt 0$. Then $G$ has subgroups of order $p^i$ for each $i$, $0\leq i\leq n$.
Proof. Induction on $n$. The result is true if $n=1$ or if $n=2$ by Cauchy's Theorem.
Assume the result holds for groups of order $p^k$, $k\lt n$. If $i=0$, we can take the trivial subgroup. If $i=1$, the result follows from Cauchy's Theorem. So assume $i\gt 1$.
Since $G$ is a $p$-group, $Z(G)\neq\{1\}$ by the class formula. Hence, $Z(G)$ has an element of order $p$. Let $g\in Z(G)$ generate a subgroup of order $p$. Then $G/\langle g\rangle$ has order $p^{n-1}$, and so has a subgroup $\overline{H}$ of order $p^{i-1}$ by the induction hypothesis. By the isomorphism theorem, $\overline{H}$ corresponds to a subgroup $H$ of $G$ that contains $\langle g\rangle$. Thus,
$$|H| = [H:\langle g\rangle][\langle g\rangle:1] = \left|\frac{H}{\langle g\rangle}\right|p = |\overline{H}|p = p^{i-1}p = p^i.$$
Thus, $G$ has a subgroup of order $p^i$. $\Box$
Alternatively, the result holds for abelian groups. If $p^i\leq |Z(G)|$, we can find a subgroup of $Z(G)$ of order $p^i$; otherwise, find a subgroup of order $p^j$ of $G/Z(G)$, where $p^j|Z(G)| = p^i$. 
A: Since $G$ is a $p$-group, $G$ has a nontrivial center by the class equation. Either the center has order at least $p^2$, and so we can find the desired subgroup inside the center by the classification of finitely generated abelian groups, or else the center has order $p$. In that case, take any other nontrivial element of $G$. If that element has order at least $p^2$, it generates a cyclic subgroup which itself has a subgroup of order $p^2$. If it has order $p$, this element will generate the desired subgroup along with the center.
A: Well, we could apply the fact that the center of such a group $G$ is nontrivial (proof). Since the center is nontrivial, it either has order $p$ or $p^m$ for $1&ltm\leq n$. In the former case, $G/Z(G)$ is of order $p^{n-1}$ so $Z(G/Z(G))$ is nontrivial, hence has order a power of $p$. Since $Z(G/Z(G))$ is abelian, it is easy to find such a subgroup (i.e. using the Fundamental Theorem of Finite Abelian Groups). The latter case is similarly easy, as $Z(G/Z(G))$ is an abelian group of $p^m$ for $m>1$.
A: Let's try a direct approach assuming the weaker condition $G$ finite with $p^2\mid |G|\quad$ (i.e., $G$ not necessarily a $p$-group):
By Cauchy, there exists an element $x \in G$ of order $p$.
Case 1: $p$ divides the index of the normalizer $\mathop{N}_G(\langle x \rangle)$ in $G$
$\langle x \rangle$ acts by right multiplication on the set of cosets $\mathop{N}_G(x)\backslash G = \{\mathop{N}_G(\langle x \rangle)g : g\in G\}\quad$. All orbits of $\langle x \rangle$ have order $1$ or $p$, implying that the number of fixed points of $\langle x \rangle$ equals the index of $\mathop{N}_G(\langle x \rangle)$ in $G$ modulo $p$, which is $0$ modulo $p$ in this case. As $\mathop{N}_G(\langle x \rangle)$ is a fixed point of $\langle x \rangle$, there has to be another fixed point, say $\mathop{N}_G(\langle x \rangle)g = \mathop{N}_G(\langle x \rangle)gx\quad$, which is equivalent to $gxg^{-1} = x^{g^{-1}} \in \mathop{N}_G(\langle x \rangle)\quad$.
As $g\not\in\mathop{N}_G(\langle x \rangle)\quad$, $x^{g^{-1}}$ is an element of order $p$ not contained in $\langle x \rangle$ that normalizes $\langle x \rangle$. Hence $\langle x, x^{g^{-1}} \rangle$ is a subgroup of order $p^2$ of $G$.
Case 2: The index of the normalizer $\mathop{N}_G(\langle x \rangle)$ in $G$ is coprime to $p$
Then the quotient group $\mathop{N}_G(\langle x \rangle)/\langle x \rangle\quad$ has order divisible by $p$. By Cauchy, it contains a subgroup of order $p$, whose preimage has order $p^2$.
