I've thought of it as: the other axioms are useful for "day-to-day math", but Replacement and Foundation are added because they make set theory more beautiful. Since the set theorists are the ones who came up with the axioms, they've earned a reward :)
Indeed, set theory without Replacement is ugly. It's interesting to see what happens in the set $R(\omega + \omega)$, the set of all sets of rank less than $\omega + \omega$, where all of ZC holds but Replacement fails. We have enough sets to do all of everyday arithmetic, analysis, etc, but we have the following facts, all of which are therefore consistent with the absence of Replacement:
Every set admits a well ordering, but many of them (e.g. $\mathbb{R}$) are not bijective to any ordinal (since all ordinals are countable). So you lose the canonical representative of a well-ordering type.
As a result, you also lose the canonical representative of a cardinality. You can't define $|A|$ as "the least ordinal in bijection with $A$" because there might not be any. So it's no longer clear what objects in your universe to use as your cardinal numbers.
Hartogs' lemma is false: every ordinal injects into $\omega$.
There exists exactly one limit ordinal, namely $\omega$. No transfinite hierarchy for you.
Ordinal arithmetic no longer works, because you cannot add $\omega$ to $\omega$, let alone multiply.
For analysis, you have $\mathbb{R}$, you have subsets and functions on $\mathbb{R}$, you have sets of sets and sets of functions, you have operators on function spaces, etc. But you can't take all those objects and put them into one single set; they are a proper class.
Even analysis can get a little tricky. Let's take tensor products of Hilbert spaces, so that $H \otimes K$ is a vector space with a bilinear map from $H \times K$, and define higher tensor products recursively as $H \otimes K \otimes L = (H \otimes K) \otimes L$. Then, unfortunately, Fock space doesn't exist, so quantum mechanics becomes awkward. (The problem was that because of the recursion, the set $H^{\otimes k}$ has a rank that increases with $k$, and so we can't put all of them in a set. We could fix it with a non-recursive definition, but I don't think most analysts want to have to take that kind of care.)
Goodstein's theorem is still true, but the proof no longer works.