By Peano Arithmetic I mean first order Peano Arithmetic. The earliest proof that it is not finitely axiomatizable that I know of is R. Montague, Semantical Closure and Non-Finite Axiomatizability I. J. Symbolic Logic 29 (1964), no. 1, 59--60. But was the result known by other means before that?
Czesław Ryll-Nardzewski, The Role of the Axiom of Induction in the Elementary Arithmetic, Fundamenta Mathematicae 39 (1952).