Any first-order logical system describes collections of objects. A set theory is a first-order logical system in which at least some of the objects of sets. In the ZFC axiomatization of set theory all objects are indeed sets. ZFC is considered to be sufficient as a foundation for (almost) all mathematics in that virtually any mathematical argument of interest can be formalized in it. This means that you can represent almost all mathematical objects of interest as sets constructed using the axioms of ZFC. There are other axiomatizations of set theory that can also do the job, but ZFC seems to be the one, if any, that mathematicians not interested in these topics learn.
(I say "almost all objects" because the collection of all sets is not a set, yet it is an object of interest, at least in category theory. This is usually handled by augmenting the set theory with a special set called a universe, whose corresponding collection of sets consists of sets called small. This then allows there to be "a categeory of small sets" whose collection of objects is a set.)