Here's a way to determine all such matrices systematically.
First, the minimal polynomial is invariant under similarity transformations. Every matrix is similar to a matrix Jordan form, therefore it suffices to look at matrices in Jordan form. Note that we can also sort the Jordan blocks from largest to smallest through similarity transformations. For $3\times 3$ matrices, it is quite easy to list all possible Jordan forms:
$$J_1 = \pmatrix{\lambda_1 & 0 & 0\\0 & \lambda_2 & 0\\ 0 & 0 & \lambda_3},
\quad
J_2 = \pmatrix{\lambda_1 & 1 & 0\\0 & \lambda_1 & 0\\ 0 & 0 & \lambda_2},
\quad
J_3 = \pmatrix{\lambda_1 & 1 & 0\\0 & \lambda_1 & 1\\ 0 & 0 & \lambda_1}$$
Now as you already deduced, the characteristic polynomial is $x^3$, therefore all eigenvalues must be zero, and the matrices reduce to
$$J_1 = \pmatrix{0 & 0 & 0\\0 & 0 & 0\\ 0 & 0 & 0},
\quad
J_2 = \pmatrix{0 & 1 & 0\\0 & 0 & 0\\ 0 & 0 & 0},
\quad
J_3 = \pmatrix{0 & 1 & 0\\0 & 0 & 1\\ 0 & 0 & 0}$$
Now as you also correctly concluded, the matrix cannot be the zero matrix; this excludes $J_1$. Also, it is easily checked that of the remaining matrices only $J_2$ has square $0$.
Therefore a matrix has the minimal polynomial $x^2$ if and only if it is similar to $J_2$.
The question remains how a matrix that is similar to $J_2$ looks like. What we immediately see is that it is of rank $1$ and has trace $0$. Since both rank and trace are invariant under similarity transformations, we can conclude that any matrix that is similar to $J_2$ must be of the form (I use $^\dagger$ to denote the conjugate transpose):
$$A = uv^\dagger \text{ with } \operatorname{tr} A = v^\dagger u = 0, u\ne 0, v\ne 0$$
On the other hand, assume that $A$ has this form. Then since the two vectors are orthogonal one can an unitary transformation that transforms $u$ to $\pmatrix{a\\0\\0}$ and $v$ to $\pmatrix{0\\b\\0}$. Thus $A$ is transformed to $\pmatrix{0 & ab^* & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0}$, which can then be further transformed to $J_2$ by the similarity transformation $TAT^{-1}$ with $T=\operatorname{diag}(1/\sqrt{ab^*},\sqrt{ab^*},1)$.
Therefore the minimal polynomial of $A$ is $x^2$ if and only if $A=uv^\dagger$ for some non-zero vectors $u,v$ with $v^\dagger u=0$.