My question is the title. I would be glad if someone could supply a proof if true, or a counterexample if false.
We assume that we are working in the predicate calculus with equality. Let $L$ be any language, and $M$ any finite $L$-structure, say with $n$ elements. Let $\Sigma$ be the set of sentences that says that there exist exactly $n$ elements, and that describes the full diagram of $M$. Then any model of $\Sigma$ is isomorphic to $M$. If the language is finite, instead of $\Sigma$ we can use a single sentence.
Remark: We consider the special case of groups, with language that has a single binary function symbol $\times$. Suppose that $M$ is a group, with elements $a_1,\dots,a_n$. In addition to the sentence that says there are exactly $n$ elements, we need as axioms that there exist $x_1,\dots,x_n$, all distinct, such that $x_i\times x_j=x_k$, for all triples $i,j,k$ such that $m_im_j=m_k$. The full diagram is unnecessary, the multiplication table is enough.
Let us spell out an example of Andre's answer (for concreteness).
Suppose a group $G$ satisfies the same first-order sentences as $C_2$, the cyclic group of order $2$.
$$\exists xy(x \neq y \wedge xx=x \wedge xy=y \wedge yx=y \wedge yy=x)$$
But this means that $G$ is isomorphic to $C_2$.