In general "the category $\mathbf{Set}$" refers to the category whose objects are all sets in whatever your chosen foundational system is. (Or perhaps only those sets that belong to some Grothendieck universe, but that's irrelevant to the point here.) If your foundational axioms are ZFC, in which all sets are pure (sets of sets of sets of ...), then the objects of $\mathbf{Set}$ are all pure sets. But if your foundational axioms are something like ZFA (ZF with atoms = urelements), then the objects of $\mathbf{Set}$ include sets of atoms. The presence or absence of atoms is generally speaking irrelevant for the categorical properties of the category $\mathbf{Set}$; in both cases it is a cocomplete well-pointed elementary topos, etc. (Other choices of set-theoretic axioms do affect the properties of $\mathbf{Set}$, but generally speaking the presence or absence of atoms does not.)
One way to make precise the question "how is the category of sets-with-atoms related to the category of pure-sets" is to work in a theory such as ZFA, where in addition to the category $\mathbf{Set}$ of all sets we can consider the category $\mathbf{Set}_{\mathrm{pure}}$ of pure sets (those which hereditarily contain no atoms). Both of these will have all the same basic categorical properties, and the inclusion $\mathbf{Set}_{\mathrm{pure}}\hookrightarrow\mathbf{Set}$ is fully faithful (since the notion of "function" is the same in all cases). If we assume the axiom of choice, then this inclusion is moreover essentially surjective, and hence an equivalence of categories; this is because under AC any set (even a set of atoms) can be well-ordered and is therefore isomorphic to a von Neumann ordinal, which is a pure set. (It's generally not sensible to ask whether two large categories are isomorphic, only whether they're equivalent.) But in the absence of the axiom of choice, there might in principle be sets containing atoms that are not isomorphic to any pure set. (For instance, I expect that probably a permutation model of ZFA contains such sets, although I'm not conversant enough with such things to give a specific example.)
With all that said, I think that your friend is asking the wrong question. There is no sense in which a mathematical set can contain real oak trees. The elements of a mathematical set, be they other sets or atoms, are other mathematical objects, not real objects that you can see, pick up, or touch. (Even a mathematical Platonist, I think, would agree that the Platonic mathematical objects exist in another Platonic world and are not things like trees that we can see and touch.) The way we use mathematics to model the real world is by choosing to represent certain real objects by certain mathematical ones. So the presence or absence of urelements in the objects of the category $\mathbf{Set}$ is irrelevant to our ability to use it to talk about oak trees; in either case we have to use some mathematical set to "label" the oak trees and thereby transfer theorems proven about the former to real facts about the latter.
