I think that, from a modern point of view, there is a misunderstanding in the position that you suggest in your question. Really, "set theory" should be understood as an umbrella term that covers a whole hierarchy of ZFC-related theories.
Perhaps one of the most significant advances in foundations is the identification of the consistency strength hierarchy. It allows us to calibrate mathematical statements with "canonical" extensions (guided by large cardinal axioms) or restrictions of ZFC. The former has been of greater interest to set theorists, so it is more visible, but the restrictions are just as important. They allow us to calibrate all sorts of things, from fragments of definable determinacy, to questions of analysis or bounded arithmetic.
In practice, many mathematicians (including many set theorists) only work in one of these restrictions, a small (or even very small) fragment of ZFC. If such a fragment suffices for your purposes, that's the one for you to use. In fact, I would argue that you should strive to work in the fragment that is appropriate for your mathematical needs, using only additional axioms beyond that fragment as your mathematics demands it.
On the other hand, some of us are interested in questions that demand significant strength beyond ZFC. For us, the appropriate theory to work on is an extension of ZFC (or NBG, or MK) with large cardinals, or a theory equiconsistent with such an extension, or a theory expected to be equiconsistent with such an extension. But also, sometimes the questions we want to look at are essentially combinatorial and $\mathsf Z_2$ (or an even weaker fragment) is the right framework in that case.
What matters is that we can go as high or as low within the ladder of the consistency strength hierarchy as the mathematical problems we encounter demand us without having to keep switching frames, so that the set-theoretic scaffolding is in many cases in the background, and only remarked upon when its presence is relevant. (For instance, questions about generic absoluteness demand constant attention to the set-theoretic framework. Questions about the partition calculus may only require a working knowledge of it.)