In his Logical Foundation of Mathematics and Computational Complexity (2013), Pavel Pudlak invites the readers to ponder about fictitious people whose natural numbers are nonstandard. His exposition is included in Section 6.1, under the subsection Interlude - Life in an Inconsistent World.
According to Gödel’s Second Incompleteness Theorem, Peano Arithmetic augmented with the formal inconsistency of Peano arithmetic, the theory that we denote by $PA + ¬Con(PA)$, is consistent. By the Completeness Theorem, this theory has a model $M$. Imagine a world $\mathcal W$ in which the natural numbers are $M$. Since $M$ is necessarily a nonstandard model, we should think of people living in $\mathcal W$ as being able to compute with nonstandard numbers like we are able to compute with standard natural numbers. I will leave to the reader’s imagination how they can do it; for example, these people may have nonstandard size, or they have standard size, but they have computers of nonstandard size and nonstandard speed, etc. This is not important for our discussion.
He goes on to say that this world is definitely fictitious, i.e. we do not live in it.
Why are we so sure that we do not live in an inconsistent world? The reason is that logic in our world seems to work perfectly; even the extremely long and complicated proofs in mathematics never fail. It is not only logic that is consistent, mathematics works perfectly as well—once a theorem is proven, it holds true in all circumstances without any exceptions. This confirms our belief in the soundness of mathematical theories. As a result we are always ready to extend our theories by statements expressing their soundness.
The problem I have with this argument is time. Presumably, it would take a non-standard amount of time to find the inconsistency in $PA$. If in our world, only a standard amount of time has passed, we would only have standard results, so of course the theorems would work. We would only notice the contradictions after a non-standard amount of time. Indeed, there could be a large initial segment of $M$ for which nothing funky goes on.
Am I missing something? How do we know we aren't in the inconsistent world?