I've been diving into Gödel's Incompleteness Theorems lately and it seems like all the weird scenarios I've encountered hinges on the existence of non-standard models of PA.
For example, if we have a consistent system S that is strong enough to express arithmetic, the statement Cons(S) = "there is no Gödel number $x$ that encodes a proof of 0 = 1" is independent of S itself. That means we can a create model where $\neg$Cons(S) is true. This is possible because in these models, there will exist a non-standard natural number that seemingly encodes a proof of 0=1.
Another example involves the Goldbach Conjecture (same logic applies to the Riemann Hypothesis). If we can prove that Goldbach is independent of PA (an unlikely scenario but let's just say that it's true), then it seems like this means Goldbach is true because we would be able to find a counterexample (i.e. an even number greater than 2 that's not the sum of two primes) in a finite number of steps if Goldbach is false. The reason why Goldbach could be independent of PA is because there are non-standard models of PA where there are non-standard naturals for which Goldbach is false.
These examples lead me to the question: what is the standard model of PA? Is it the von Neumann ordinals, given by the axiom of infinity? How do we decide which model is standard and which is non-standard? Is there actually a collection of standard models, which are equivalent up to an isomorphism?