We know that order of any element of a group divides the order of the group. So possible orders of elements of our are 1, 2, 4. Moreover, only identity has order equal to 1. So all other elements must have orders 2 or 4.
If there is an element of order 4, this group is cyclic.
So the only remaining case is that there might be a group with four elements where all non-identity elements are of order two.
We know that every group with this property is commutative, see Prove that if $g^2=e$ for all g in G then G is Abelian. or Order of nontrivial elements is 2 implies Abelian group
But for the case of 4 elements, we can also find this group by filling
out the Cayley table.
Let us denote the elements of the group $G=\{e,a,b,c\}$, where $e$ is
the identity. So far we know that $a^2=b^2=c^2=e$.
$$\begin{array}{|c|cccc|}
\hline
& e & a & b & c \\\hline
e & e & a & b & c \\
a & a & e & & \\
b & b & & e & \\
c & c & & & e \\\hline
\end{array}$$
Now we can continue in a sudoku-like fashion. From the cancelation law we know that we cannot have the same element twice in the same row/in the same column. (If an element appears twice in a row $x$, it means that $x*y=x*z$ for $y\ne z$. Cancelation law says that this cannot happen in a group.)
So, for example, $a*b$ equals neither $a$ nor $e$, since this would mean having repeated letter in a row $a$. But it cannot be equal to $b$ either, since $b$ already is in this column. So the only remaining possibility is $a*b=c$. Almost in the same way we get that $b*a=c$.
$$\begin{array}{|c|cccc|}
\hline
& e & a & b & c \\\hline
e & e & a & b & c \\
a & a & e & c & \\
b & b & c & e & \\
c & c & & & e \\\hline
\end{array}$$
Now we continue like this - if there is only one possibility, we can write down an element to this position. We can fill the whole table.
$$\begin{array}{|c|cccc|}
\hline
& e & a & b & c \\\hline
e & e & a & b & c \\
a & a & e & c & b \\
b & b & c & e & a \\
c & c & b & a & e \\\hline
\end{array}$$
So we see that if a group on 4 elements is not cyclic, then it must have table like this. This table is symmetric w.r.t. the main diagonal, which means that the group is commutative.
Remark 1: So far we do not know whether the above table determines a group. We only know that if a non-cyclic group on 4 elements exists, it must have table like this. It can be shown that this is indeed a group. A very laborious way to do this would be checking associativity for all triples. But if we already know about the group $(\mathbb Z_2\times\mathbb Z_2,\times)$, it is not difficult to notice that it has "the same" table.
$$\begin{array}{|c|cccc|}
\hline
& (0,0) & (0,1) & (1,0) & (1,1) \\\hline
(0,0) & (0,0) & (0,1) & (1,0) & (1,1) \\
(0,1) & (0,1) & (0,0) & (1,1) & (1,0) \\
(1,0) & (1,0) & (1,1) & (0,0) & (0,1) \\
(1,1) & (1,1) & (1,0) & (0,1) & (0,0) \\\hline
\end{array}$$
Remark 2: We have in fact proved something more that was asked. We have not only shown that every group on 4 elements is commutative. We have also shown that it is either cyclic or has the table described above. This means that there are only two groups having 4 element "up to isomorphism" (=all other groups have the same table, only elements are "renamed"). Namely, every group with $|G|=4$ is isomorphic either to $(\mathbb Z_4,+)$ or to $(\mathbb Z_2\times\mathbb Z_2,\times)$.