How exactly do we explain "distinct objects" in the definition of set? Wikipedia gives the following definition for sets:

A set is a well-defined collection of distinct objects.

But what does it mean by "distinct objects" here? 
For example, we can say $\{A,B\}=\{\{1,2\},\{1,3\}\}$ is a set. So essentially by "distinct" we wanted to say $A\neq B$. 
However, I think this is not enough. For example, can we say $\{\{a\},a\}$ is a set? I guess not. Just consider $\{$America, New York$\}$. It would be inconvenient to discuss problems when we put objects in different level of hierarchy together. So we should have $A\notin B$ and $B\notin A$ to say $A,B$ are distinct. But how about $\{\{\{a\},\emptyset\},a\}$?
 A: In fact $\{\{a\},a\}$ and $\{\{\{a\},\emptyset\},a\}\{\{\{a\},\emptyset\},a\}$ are indeed both examples of sets. Namely, in set theory, every set is an object as well. The 'hierarchy' as you call it depends interely on the context. Your example of $\{\text{America}, \text{New York}\}$ can be seen as a set of two English words. Or you can regard 'America' as the set of all cities in America, in which case New York $\in$ America, i.e. 'New York' is an element of the set 'America'.
A: In modern set theory, there are no types. Everything is a set. This means that all mathematical objects in the mathematical universe—as far as set theory is concerned—are sets. You are thinking about "types", and we can model the notion of types within set theory. But the set theoretic universe itself is blind to that, just as much as your CPU being blind as to the code being run was written in Java, Lisp, C++, or D (it might know it's Cobol and refuse to run it, though).
Even if you do work in a set theory which allows for non-set objects, there is still no restriction on sets having both set and non-set elements. This is something you'd see often in a first course in set theory. People don't understand (at first) that sets can be elements of other sets, and that sets can have all sort of elements.
Now, if we want to talk about formal definition of "distinct", it just means not equal. Equality is "built in" to the universe, so different objects are different. The axiom of extensionality tells us that sets are different when we can detect that with different elements. Namely, $\{0,1\}$ is distinct from $\{1,2\}$ because $0$ is an element of the one set but not the other. In the case we allow for non-sets, again, equality is something we just get from the rules of logic.
