All subgroups of a abelian group are normal. But the converse is not true. If every subgroup of a group is normal, then what more can we say about the group?
If $G$ is a finite non-abelian group where all subgroups are normal, then $$G \cong Q_8 \times A \times B$$ where $A$ is an elementary abelian 2-group (ie, all non-identity elements have order 2), $B$ is abelian of odd order and $Q_8$ is the quaternion group of order 8. A proof can be found in for example Berkovich's Groups of Prime Power Order I believe.