By Löwenheim-Skolem we know there are models of (first order) PA that are not isomorphic to the standard model, but are elementary equivalent to it, i.e. they satisfy the same set of first-order sentences.
By Gödel's incompleteness result, we know there is a model of PA in which the canonical unprovable Gödel sentence G is true (the standard model), and (non-standard) models where G is false.
Since G is a first-order sentence, this seems to answer the question I posed in the title to the negative. Correct so far?
I am wondering then, is it a meaningful, i.e. well defined question to ask if for any "other" first-order sentences, those that intuitively express more naturally "arithmetic truths", all models of PA are elementary equivalent? are there other more arithmetically meaningful$^1$ (first order) sentences that are independent from the PA axioms?
$^1$ where 'more ... meaningful' is probably too vague to hope for an answer, but I am essentially asking, can we be sure that any first-order property we hold as evident of $\mathbb{N}$ is provable from the axioms of PA?