I can't speak to heuristics (and I suspect very little is known there -
these are very complex and rare semirings!), but here's one surprising appearance of a nonstandard model of true arithmetic:
A set is Dedekind-finite if it has no nonsurjective self-injections (equivalently, if it has no countably infinite subset). With the axiom of choice, Dedekind-finiteness is the same as finiteness; however, in the absence of choice this is not true. And the equivalence can fail spectacularly - for instance, by having amorphous sets, which are infinite sets which cannot be partitioned into two infinite pieces!
With choice, the cardinalities are well-behaved (in particular, linearly ordered); without choice, they can be extremely messy. So it's interesting to ask what kind of nice behavior is compatible with having bad "pseudo-finite" sets?
Specifically, we can ask the following question:
Is it consistent with ZF that there are infinite Dedekind-finite sets, and that the set of Dedekind-finite cardinalities is linearly ordered?
That last clause just means that given any two Dedekind-finite sets $A$ and $B$, either $A$ injects into $B$ or $B$ injects into $A$.
Sageev showed, in a technical and difficult paper, that the answer is yes; but for our purposes, the interesting result is earlier and due to Ellentuck: that in such a model, the set of Dedekind-finite cardinalities (together with disjoint union and Cartesian product) forms a nonstandard model of true arithmetic! Put another way, models of set theory satisfying some mild-seeming cardinal arithmetic axioms turn out to have distinguished nonstandard models of arithmetic!
A neat corollary of this is that in ZF, if the Dedekind-finite cardinalities are linearly ordered then there are no amorphous sets; this can be proved by invoking, for instance, the fact that every natural number is either even or odd! And I don't know an easy way to explicitly build two incomparable Dedekind-finite sets from an amorphous set, so I don't think this is as silly a nuke as it may seem.
To me this is completely surprising, especially given that the definition of the model is so simple. And my understanding is that Ellentuck's result was quite surprising in general; in his own paper, Sageev refers to the result as "astonishing," and mentions that its proof uses basically all of the combinatorial results developed in Ellentuck's thesis from twelve years earlier. Ellentuck's paper is titled "A model of arithmetic," and was published in the Bulletin of the Polish Academy of Science (at least I think; the citation in Sageev's paper is to "Bull. Acad. Polon. Sci.") in 1974, but I have sadly not been able to find a copy of it online.
Interestingly, the overall structure of the "Ellentuck models" of arithmetic seems potentially rich but not yet well-understood; see this question of mine. (It's also worth noting that Sageev's proof of his result required an inaccessible, but it is not known whether that is necessary.)