Normal subgroup of order $p$ is in the center Prove that in a group of order $p^2$ ($p$ a prime), a normal subgroup of order $p$ lies in the center.   (Since this is Exercise 2.9.6 from Herstein's Topics in Algebra, there are some restrictions on what I can use.  This section is on Cayley's theorem.  I can show, using the class equation, that $Z(G)>1$ and $G$ is abelian, but I don't want to use the class equation yet.)  
If $H$ is a subgroup of order $p$, since $|G|$ does not divide $i(H)!$, $H$ contains a nontrivial proper subgroup $K$ of $G$. Since $p$ is a prime, $K$ must equal $H$.  Hence $H$ is normal.  Why is it in the center?  
 A: If $G$ were cyclic it would be abelian, and we’d be done. Suppose it’s not. Then every element is of order $p$ by Lagrange’s theorem. And every subgroup of size $p$ is normal by Herstein’s lemma 2.9.1.
Subgroup $H$ must be cyclic having elements of order $p$. Then any two elements of $H$ commute.
Suppose $k\not\in H$. Then $k$ generates a group $K$ of size $p$, and this group is normal (using the first paragraph).
Now $H\cap K=\{e\}$ since it’s a subgroup of size $<p$. Then, for any $h\in H,$ $hkh^{-1}k^{-1} \in H\cap K$ using the normalcy of both subgroups (eg $hkh^{-1}\in K$ since $K$ is normal, and so on). But then
$$hkh^{-1}k^{-1} = e \Rightarrow hk = kh.$$
A: This type of exercises are difficult to judge in a forum like this, because we need to know precisely, what has been covered up to that point. Many posters are then inclined to use extra bits they have learned (later in a similar course).
I get the feeling that the following is what might have been expected. This is just fleshing out the hint in Steve D's comment, so I make it a CW.
Let $H$ be a normal subgroup of order $p$. We know (Lagrange's theorem) that $H$ is cyclic. Let $g$ be a generator. Let $x$ be any element of $G$. By normality of $H$ we know that
$$
xgx^{-1}=g^k,
$$
for some integer $k, 0<k<p$. Conjugation by $x$ is an automorphism of $H$, so
$xg^{t}x^{-1}=g^{tk}$ for all integers $t$. In particular we get that
$$
x^2g x^{-2}=x(xgx^{-1})x^{-1}=xg^kx^{-1}=g^{k^2}.
$$
An obvious induction then proves that
$$
x^tgx^{-t}=g^{k^t}
$$
for all natural numbers $t$. But, again by Lagrange's theorem $x^{p^2}=1$. Therefore
$$
g=1g1^{-1}=g^{k^{p^2}}.
$$
As $g$ is of order $p$, this means that $1\equiv k^{p^2}\pmod p$. But two applications of Little Fermat tell us that
$$
k\equiv k^p \equiv k^{p^2}\equiv 1\pmod p.
$$
Recalling the constraint $0<k<p$ we can conclude that $k=1$. Therefore $x$ and $g$ commute. Obviously then $x$ commutes with all the powers of $g$. As $x$ was arbitray, we have shown that $H\le Z(G)$.
A: If $H\not\subset Z$ then  $H\cap Z=1$. Hence $HZ$ is subgroup of order $\ge p^2$, i.e. $HZ=G$ and $G$ is Abelian.
A: In fact it must be $\,Z(G)=G\Longleftrightarrow G\,$ is abelian, in this case. You only need the easy
Lemma: For any group $\,G\,$ , the quotient $\,G/Z(G)\,$ cannot be cyclic non-trivial.
The above says that $\,G/Z(G)\,\,\,\text{cyclic}\;\Longleftrightarrow \;G\,$ is abelian...
A: Let $H$ be the normal subgroup of size $p$. By Langrange's theorem $H$ must be generated by an element say $g \in G$ s.t. $g^p=e$. 
Since $H$ is normal in $G$, $G/H$ is a group and has order $p$. Let this group be generated by $Hb$. Then $e \neq b \notin H$ and $Hb^p =H$. We assume here that $b^p=e$ otherwise order of $b$ is $p^2$ and $G$ is generated by $b$ meaning $G=Z(G)$.
At this point it is clear that $G$ is generated by $a$ and $b$ because $G$ can be partitioned into cosets of $H$ that have elements that are products of $a$ and $b$.
$H$ is normal in $G \implies bab^{-1} = a^i$ (for some $i$). $\iff ba= a^ib$. 
Consider a subgroup $H_2$ generated by $b$ in $G.$ Since $b \neq e$ and $b^p=e$, $o(H_2)=p$. 
As $|G| =p^2$ does not divide $i(H_2)! = p!$ $H_2$ must be normal in $G$.
$H_2$ normal in $G \implies a^{-1}ba = b^j $ (for some $j$) $\iff ba = ab^j$.
$ba = a^ib$ and $ba = ab^j$ imply that $i=j=1$ and $ba = ab$.
Thus $G$ is abelian or $G=Z(G)$.
