Can we avoid an axiomatic theory of sets by never formulating paradoxes? We know that ZFC was formulated to avoid some paradoxes inherent to Cantor's naive set theory, such as Russell's paradox, which inquires about the truth of the existence of the set of all sets.
The notion of set, however, comes prior to any axiomatic set theory: to write ZFC's nine axioms, we are using sets of variables, constants, functions and relations. Those primitive, pre-ZFC sets, don't generate paradoxes simply because we choose never to write paradoxes with them: those sets are solely used to generate ZFC.
The point is: can we avoid ZFC in its entirety by keeping on formulating mathematics with those primitive sets and never touching any paradoxes?
 A: The question reads equivalent to "Will my hard drive always have free space, as long as I don't check that?", and the answer is of course not. If there is a paradox, then you will run into it or at some point be convinced that it is unlikely to exist.
But if we look deeper into your question, then we can look at it as a question asking about where do the axioms of $\sf ZFC$ "live" themselves.
The answer can be looked at from three main point of views:


*

*There is a big universe of sets, and we believe that it satisfies the axioms of $\sf ZFC$ and devoid of paradoxes (at least in the Russell-Cantor-Burali Forti kind of paradoxes).
Then we formulate some axioms which seem "reasonable" and we want to investigate models of those axioms which are "toy universes" if you will, and in the process learn about truth and false statement in the real universe of sets.

*There is no universe of sets. We have the natural numbers, a gift given to us by Brouwer's god. Using clever tricks, like Godel coding, we develop first-order logic using the natural numbers, and we have a formula which states when something is an axiom of $\sf ZFC$. And everything we do, we do in the confines of finding a proof or refutation of statements from these axioms.
But these are manipulations of natural numbers. There are no sets, there is no magical bean or some figure behind the curtain. We have the natural numbers, and we work there.
Of course, you might feel obligated to ask, where did the natural numbers come from? Aren't they a product of the universe of sets? Yes, this is one option. Another is to related claim that all we do is write a few strings on paper and manipulate them.
These are philosophical questions and approaches to mathematics, and you have to figure it out on your own. Oh yeah, I promised three approaches.


*Everything about infinity is bollocks, and we waste our time in modern set theory and modern mathematics. Okay, this one is not as popular as the others. And it's a good thing too, I guess.

