As per usual, let PA denote Peano Arithmetic and ZFC denote Zermelo-Fraenkel set theory with choice. Furthermore, ZFC 'validates' PA, in the sense that it proves that the PA axioms hold for the standard model of the natural numbers. Arguably, PA also validates ZFC, albeit in a weaker sense. In particular, note that, if ZFC is consistent, then the sentences it proves about the natural numbers do not contradict the theorems of PA.
However, I'm wondering: can we find theories PA' and ZFC' that contradict the more usual PA and ZFC, but which validate each other in much the same way that PA and ZFC validate each other? Obviously the answer is 'yes', but what if we further require that PA and PA' agree on on all $\Pi_1$ sentences and all $\Sigma_1$ sentences, and similarly that ZFC and ZFC' agree on all $\Pi_1$ sentences and all $\Sigma_1$ sentences in the language of arithmetic?
Then we'd have no 'trial-and-error' method for choosing between the foundation (PA,ZFC) versus the foundation (PA',ZFC'), since it is only $\Pi_1$ and $\Sigma_1$ sentences in the language of arithmetic that can actually be checked in a mechanical fashion. More precisely, the $\Pi_1$ sentences of arithmetic are precisely those that can be mechanically falsified (if false), while $\Sigma_1$ sentences are those that can be mechanically verified (if true).
So is this possible?
And if so, how do we decide which is the correct, true math? (And, is this even a coherent idea?)
For instance, it is known that the twin-prime conjecture is $\Pi_2$. What if the PA/ZFC approach to math proves that the twin-prime conjecture is true, while the PA'/ZFC' approach proves that its false? What then?