The littlest counter-example :
$$Aut(\frac{\mathbb{Z}}{2\mathbb{Z}}\times \frac{\mathbb{Z}}{2\mathbb{Z}})=S_3 $$
All three non-zero elements are freely permuted.
Now, with such a question, you have two ways, either you take a non-commutative group with trivial center, in which case you will have $G=G/Z(G)=Inn(G)$ hence a non commutative subgroup of $Aut(G)$ or you can also consider the following property which is not hard to show: A $\underline{\text{finite}}$ Abelian group $A$ has an abelian automorphism group if and only if $A$ is cyclic.
Something less intuitive : There exists non-abelian groups with abelian automorphism group.
The proof of $A$ abelian non cyclic implies that $Aut(A)$ is non commutative goes as follows : we know that an abelian group $A$ is the direct product of its $p$-Sylows :
$$A=S_1\times...\times S_r$$
Clearly (it should be checked, the $p$-Sylows are characteristic) this leads to :
$$Aut(A)=Aut(S_1)\times ...\times Aut(S_r)$$
Hence, it suffices to check the property for abelian $p$-groups. Now assume $A$ is an abelian $p$-group and that $A$ is non-cyclic then we have that :
$$A=\frac{\mathbb{Z}}{p^{a_1}}\times...\times \frac{\mathbb{Z}}{p^{a_r}}$$
and $r\geq 2$. Now define :
$$B_1=\frac{\mathbb{Z}}{p^{a_1}}\times...\times \frac{\mathbb{Z}}{p^{a_{r-2}}}$$
$$B_2=\frac{\mathbb{Z}}{p^{a_{r-1}}}\times \frac{\mathbb{Z}}{p^{a_r}}$$
We have that $Aut(A)$ contains $Aut(B_1)\times Aut(B_2)$. I will show that $Aut(B_2)$ is non commutative. Define $e_1$ and $e_2$ to be respectively the element $(1,0)$ and $(0,1)$ of respective orders $p^{a_{r-1}}$ and $p^{a_r}$. Now you have two automorphisms :
$$\phi_1\text{ defined by }\phi_1(e_1):=e_1\text{ and } \phi_1(e_2):=e_1+e_2 $$
$$\phi_2\text{ defined by }\phi_2(e_1):=e_1+p^{a_r-a_{r-1}}e_2\text{ and } \phi_1(e_2):=e_2 $$
I claim that this defines group morphisms (it suffices to check that $\phi_i(e_1)$ is of order $p^{a_{r-1}}$ and $\phi_i(e_2)$ is of order $p^{a_{r}}$). Secondly they are surjective hence bijective so they are group automorphisms of $B_2$. Now :
$$\phi_1\circ\phi_2(e_2)=e_1+e_2 $$
$$\phi_2\circ\phi_1(e_2)=\phi_2(e_1+e_2)=e_1+(1+p^{a_r-a_{r-1}})e_2 $$
Clearly we have : $\phi_1\circ\phi_2\neq \phi_2\circ\phi_1$. Hence we defined two non-commuting automorphisms which imply that $Aut(B_2)$ is non commutative hence $Aut(A)$ is non commutative as well.