I often read that arithmetic in first order logic has problems and you really want to do it in second order logic.
However, aren't the Zermelo–Fraenkel axioms written down in the language of first order logic?
Note that ZFC is a theory strong enough to prove second-order arithmetics. So if you agree to take ZFC as your foundational point, taking second-order PA for arithmetic should not pose any problems.
This is one of the reasons set theory is a good foundational basis for mathematics, since it allows second-order (and higher) to work via first-order formulas in the universe of set theory.