If we look at the axioms of Peano arithmetic, e.g. http://mathworld.wolfram.com/PeanosAxioms.html, they contain an axiom:
If $a$ is a number, the successor of $a$ is a number.
However, the axioms do not limit how many times we could apply this successor operation. So we could apply successor to $0$ finitely many times, infinitely countably, uncountably, etc. So these axioms cannot define natural numbers because all the numbers generated would not be bijective with the natural numbers.
Is it a standard mathematical assumption that axioms can be applied only finitely many times to derive a sentence or construct a mathematical object? In set theory, there is an axiom of infinity, although we could construct any infinite set within infinite number of steps (applications if the other axioms).
When defining a successor, we apply it once and get a number $1$ from $0$. But how can we apply it once if we have only a definition of $0$ available? Is this not a problem?