In ZFC, why was it chosen to not define what exactly a set or $\epsilon$ relation is? From about the first twenty minutes of this lecture by Frederic Schuller, what I understood was that the philosophy of ZFC was that we define the idea of a set and $\epsilon$ relation by their properties rather than stating what they are literally.
However, why exactly do we not define these objects? What exactly is the difficulty to do this?
 A: What something "is" is rarely as important or meaningful as how it behaves. Consider similar questions about less mysterious structures, like "what are the natural numbers", or "what is the trivial group". In each case set theory has an "answer" in the form of a natural construction:

*

*The natural numbers are given by $0:=\emptyset,$ $1:=\{\emptyset\},$ $2:=\{\emptyset, \{\emptyset\}\},$ and so on. (And if we want, the usual operations and relations can be defined recursively on the set of all natural numbers.)


*The trivial group is the structure $(G,e,\cdot)= (\{\emptyset\}, \emptyset, \{((\emptyset, \emptyset),\emptyset)\})$
But do these constructions really give us any useful insights? More to the point, are they really any kind of answer for these things are?
This is not to say that set theoretical constructions are useless. Maybe we're dealing with something slightly more complicated where there's a real worry about existence. For example, the construction of the real numbers as Dedekind cuts might convince a skeptic that complete ordered fields really do exist. Same goes with the first uncountable ordinal. But the point is that the main technique we have for saying what something "is" is to give a set theoretical definition, and at least in the simplest cases, this is usually really only elucidating insofar as it convinces of the consistency (relative to set theory) of the underlying idea.
(Also, it's not my intention to criticize set-theoretical foundations in general along these lines. This is sometimes used as a talking point to that end, but I don't really agree with that criticism.)
Moreover, although certain constructions are more natural than others, they are never unique. The best we can have is to have a certain list of properties that defines a structure uniquely up to isomorphism. We can always swap out the "building" blocks with some other sets and keep the relationships the same. Sometimes there are even multiple natural ideas for the construction that all work: are the real numbers Dedekind cuts of rationals or equivalence classes of Cauchy sequences of rationals? Does it matter?
So, going back to your question, if we wanted to have a definition for what a set is, we'd need to have some other methodology for mathematical constructions from which we could define them. We could use usual methodology (set theory) but that seems circular for these purposes. (Although in general it's unproblematic to construct set theoretical universes using set theory, aside from the fact that, per Godel's incompleteness theorem, you need to assume stronger axioms than those that hold in the universe you're constructing.) There's no question that this could be done, perhaps even in a way that isn't totally contrived, but the question is what would it get us?
A: The short answer is that we do not understand well enough what a set is.
There is a definition of a set which goes back to Cantor:

A set is a gathering together into a whole of definite, distinct objects of our perception or of our thought—which are called elements of the set.

Unfortunately, this description does not seem precise enough to decide whether some of the more exotic collections of objects - the collection $V$ of all sets, the collection $Card$ of all cardinals, the Russell class $R=\{x|x\notin x\}$ - are sets or not. Cantor later came up with the term inconsistent multiplicities to rule out collections such as $Card$ whose "gathering together into a whole" implies a contradiction, but arguably also this extended notion is too imprecise to really nail down the concept of a set.
Instead of such "philosophical" definitions of sets, we now have axiomatic theories which describe some properties of sets which are deemed uncontroversial. This allows one to just work with the properties, instead of building on some potentially nebulous intuition about what sets "really are".
One such theory is ZFC. It is well-known that ZFC is incomplete, and therefore allows for different models - each of which has a different concept of set. Now if you do believe that there is one and only one true concept of set, then the incompleteness of ZFC means that there must be some true set-theoretic principles that we have overlooked so far. Note however in this context that due to the results of Gödel (and Turing), no complete theory of sets can be computably enumerable.
