As you note in the question, the group of inner automorphisms Inn($G$) is isomorphic to $G/Z(G)$. In particular, it's trivial if and only if $Z(G)=G$. So there is a non-trivial (inner) automorphism unless $G=Z(G)$.
Now, notice that, by definition, $Z(G)=G$ if and only if $G$ is abelian; so we have reduced to the abelian case.
If $G$ is abelian then $g\mapsto -g$ is an automorphism, and it is non-trivial unless $g=-g$ for all $g\in G$. But $g=-g$ if and only if the order of $g$ divdes two. So we have now reduced to the case in which $2g=0$ for all $g\in G$.
In this case, $G$ is a vector space over the field $\mathbb{Z}/2$. As
$|G|$ is equal to 2 raised to the power of the $\mathbb{Z}/2$-dimension of $G$,
the hypothesis that $|G|>2$ implies that $\mathrm{dim}_{\mathbb{Z/2}} G>1$. But now we can write down lots of linear automorphisms of $G$. For instance, you could fix any basis $g_1,g_2,\ldots$ and take the automorphism $g_1\mapsto g_2$, $g_2\mapsto g_1$ and $g_i\mapsto g_i$ for every $i>2$.