In mathematics definitions (and axioms) are the attempts to formalize some informal notions.
Sets come to formalize the notion of a "collection", so we can talk mathematically about collections of objects. The collection makes a distinction between two things which are not equal, but that's it. So if I open my wallet, and look at my coins, while I might have two coins of the same value, they are not the same coin.
Why this notion and not the notion of a multiset, where we also care about repetition? Because we want something bare, with as least structure as possible. You can always add structure to things which don't have any, but you can't remove structure from your atomic notion. (For example, a field is a ring, is an abelian group, is a group, is a set. But if the most basic objects in your world is a field, you can't strip it from structure anymore.)
In modern terms, sets are objects of a universe of set theory. It may sound circular, but only at the level of natural language where I used the term "set theory" to define "set". Where set theory is an informal, but rather well-understood term for theories whose concern is formalizing the notion of set into a mathematical object.
And why do we want them to be with the least structure possible? Because using the axioms of set theory we can prove that we can endow them with pretty much any structure we want (well, up to a certain limitation, but certainly we can endow them with the structure of a multiset).