The difference between Z and ZF Wiki says that Goodstein's theorem is provable in ZF, since we have ordinal theory in ZF. 
My question is, is the axiom of replacement necessary in the proof? (which is the only axiom that occurs in ZF but not in Z) Could we construct the whole ordinal theory without the axiom schema of replacement? 
 A: The axiom of replacement is not necessary for the proof.  The axiom of replacement is needed for certain aspects of the theory of the ordinals, but the proof of Goodstein's theorem just needs a certain relatively small countable well-ordered set, which can be done in Z using a certain well-ordering of the natural numbers.
To explain a little more, the axiom of replacement is needed to construct the von Neumann ordinal associated to any well-ordered set.  The von Neumann ordinals are defined so that each ordinal is the set of all the previous ordinals, so that the order relation on them is just the element relation $\in$.  This is useful for various technical purposes, but is not at all necessary for most applications of ordinals, since usually all you need is a well-ordered set with a desired isomorphism type.  The axiom of replacement is also needed to construct certain very large (from the perspective of "ordinary" mathematics) sets that are needed in certain arguments.  For instance, you can't prove that $\mathbb{N}\cup\mathcal{P}(\mathbb{N})\cup\mathcal{P}(\mathcal{P}(\mathbb{N}))\cup\dots$ is a set without repacement.  Nothing of this sort is needed for Goodstein's theorem or most other applications of ordinals, though.
