Existence of a group of order $ p^p $ Let $ p $ be a prime number. Does there exist a group $ G $ of order $ o(G)=p^p $ such that all elements in $ G $ different from the identity element have order $ p $ ?
Motivation for the question : An example for $ p=2 $ is obvious, the Klein 4-group. 
For $ p=3 $, I came across this very nice example. Let $ G = U_3( \mathbb{F}_3 ) $, the group of all upper triangular matrices over $ \mathbb{F}_3 $, with all entries on the main diagonal equal to $ 1 $. Note that $ o(G) = 27 $ and for any $ M \in G, M \neq I $, we can write $ M=I+N $, $ N \neq 0 $. Note that $ N $ is a strictly upper triangular matrix, hence nilpotent, $ N^3 = 0 $. From this, we get $ M^3 = (I+N)^3 = I + N^3 = I $ and hence all elements other than the identity are of order $ 3 $.
As pointed out below, $ G = (\mathbb{Z}_p)^p $ does the trick. For $ p \ge 3 $, does there exist an example which is not isomorphic to $ (\mathbb{Z}_p)^p $?
 A: Take $p>2$ a prime number. Define your group :
$$G:=F\rtimes_{\phi}C_p$$
Where $F=C_p^{p-1}$ and $C_p$ denotes the cyclic group of order $p$ with generator $1$. Define the action of $\phi$ by :
$$\phi(1).(a_1,...,a_{p-1}):=(a_1+a_{p-1},a_2,...,a_{p-1})$$
Then I claim that this is well defined (this correspond to a group morphism from $C_p$ to $Aut(F)=GL_n(\mathbb{F}_p)$). Furthermore any element of $G$ is either in $F$, $C_p$ or is in the group generated by an element of the form $(x,1)$. In the first two cases we, (of course) get elements of order $p$.
In the third case :
$$(x,1)(x,1)=(x+\phi(1).x,2)=(2x+(x_{p-1},0,...,0),2) $$
$$(x,1)^3=(3x+(1+2)(x_{p-1},0,...,0),3) $$
Hence by induction :
$$(x,1)^k=(kx+\frac{k(k-1)}{2}(x_{p-1},0,...,0),k) $$
Remark that the notation makes sense since $2$ is invertible mod $p$. Hence for $k:=p$ we get that :
$$(x,1)^p=(px+\frac{p(p-1)}{2}(x_{p-1},0,...,0),p)=(0+p(\frac{p-1}{2}x_{p-1},0,...,0),0)=(0,0) $$
Hence, on the whole, any element is of order $p$. Since $G$ is clearly non abelian we have our group.
A: The group of upper triangular matrices is interesting than symmetric group, since $p$-groups sit in very interesting manner in the uppper triangular group than in symmetric group. Therefore, I would give an example from group of "matrices". 
We know that every odd prime $p$ can be written as sum of two consecutive numbers (if $p=4n+1$ the $p=2n+(2n+1)$ and if $p=4m+3$ then $p=(2m+1)+(2m+2)$.
Let $p$ be an odd prime, and write $p=k+(k+1)$. Consider set of following matrices over $\mathbb{Z}_p$:
$$
\begin{pmatrix}
1 & a_{1} & a_{2} & \cdots & a_{k} & a_{k+1}\\
0 & 1 & b_{1} & b_{2} & \cdots & b_{k}\\
0 & 0 & 1 & 0 & \cdots & 0\\
0 & 0 & 0 & 1 & \cdots  & 0\\
0 & 0 & 0 & 0 & \ddots  & 0\\
0 & 0 & 0 & 0 & \cdots  & 1\\
\end{pmatrix}  (a_i,b_j\in \mathbb{Z}_p)
$$
This is a (non-abelian) group of order $p^{k+(k+1)}=p^p$ with usual matrix multiplication. By elementary linear algebra, we can show that each element has order $p$. Since, by Jordan's theory, any non-identity element in this group is conjugate to, 
$$
\begin{pmatrix}
1 & 1  &  &   &  &  \\
 & 1 & 1 &  &  & \\
 &  & 1 &  &  & \\
 &  &  & 1 &   & \\
&  &  &  & \ddots  & \\
 & &  &  &   & 1\\
\end{pmatrix}   \mbox{ or } 
\begin{pmatrix}
1 & 1  &  &   &  &  \\
 & 1 &  & &  & \\
 &  & 1 &  &  & \\
 &  &  & 1 &   & \\
 &  &  &  & \ddots  & \\
 &  &  &  &   & 1\\
\end{pmatrix}
$$
these matrices have minimal polynomial $(x-1)^3$ or $(x-1)^2$ which will certainly divide $(x-1)^p$ since $p\geq 3$. Thus, For every matrix $A$ in $G$, which is conjugate to one of above two matrices, we have $(A-I)^p=0$, and since matrix entries are in $\mathbb{Z}_p$, we have $A^p-I=0$, i.e. $A^p=I$. Thus, every matrix in above group has order $p$.
