Note: According to OPs formulation in the bounty text this answer is aimed to give at least a glimpse of some aspects around the definition of set and set theories with focus on the axiom of choice. The top voted answers already contain the essential information.
On the definition of sets:
In modern times sets (and all other mathematical objects) are defined not in order to specify what they are but instead what we want to do with them. This led to different set theories which emerged essentially due to three different strands and common to all of them was the goal to develop a substantial mathematical theory.
Set theory as a tool in understanding the infinite leading to the theory of cardinals and ordinals. We owe Georg Cantor the fundamentals of this theory who did it more or less single-handed and against all odds.
Set theory as foundation of supplying the subject matter of mathematics. This claim reflects the mainstream and in many books we can find something like set theory is the foundation of mathematics.
Set theory as supplier of a common mode of reasoning for diverse areas of mathematics. This is strongly related with the second strand and the axiom of choice is a famous set-theoretical principle of this type.
This reasoning from M. Potter's Set Theory and It's Philosophy is followed by
The history of such theories is now a century old ... - and yet there is, even now, no consensus in the literature about the form they should take.
Conclusion: There is no widely agreed single set theory which is favored by the mathematicians from which we could derive a right definition of a set. Instead depending on the area of research and the richness of results within these areas different set theories like ZF, ZFC, etc. are taken as their basis.
Set theory today:
Presumably most of the daily work is explicitly or implicitely based upon ZF, ZFC or a somewhat weaker version in between. But we should be aware that each of these set theories has benefits as well as drawbacks.
U. Felgner writes in Models of ZF-Set theory
We believe that the ZF-axioms describe in a correct way our intuitive contemplations concerning the notion of sets. The axiom of choice (AC) is intuitively not so clear as the other ZF-axioms are, but we have learned to use it because it seems to be indispensible in proving mathematical theorems. On the other hand (AC) has strange consequences, such as every set can be well-ordered and we are unable to imagine a well-ordering of the set of real numbers.
Besides the well-order theorem (WOT) there are many other equivalents to the AC which are then also to accept.
Historical aspects around AC:
Some mathematicians had difficulties deciding for or against this axiom e.g. van der Waerden:
In 1930, van der Waerden published his Modern Algebra, detailing the exciting new applications of the axiom. The book was very influential, providing Zorn und Teichmüller with a proving ground for their versions of choice, but van der Waerden's Dutch colleagues persuaded him to abondon the axiom in the second edition of 1937. He did so, but the resulting limited version of abstract algebra brought such a strong protest from his fellow algebraists that he was moved to reinstate the axiom and all its consequences in the third edition of 1950. (P. Maddy, 1988)
H. Herrlich summarizes the historical development in his Axiom of Choice
After Gödel (1938) proved the relative consistency of the Axiom of Choice by constructing within a given model of ZF a model of ZFC, the proponents of AC gained ground. Most modern textbooks take AC for granted and the vast majority of methematicians use AC freely.
However, after Cohen (1963) proved the relative consistency of the negation of AC and, moreover, provided a method, called forcing, for producing a plethora of models of ZF that have or fail to have a wide range of specified properties, a growing number of mathematicians started to investigate the ZF world by substituting AC by a variety of possible alternatives, sometimes just by weaking AC and sometimes by replacing AC by axioms that contradict it.
And with respect to a true definition of sets he continues
All this work demonstrates how useful or convenient such axioms as AC and its possible alternatives are. But the question of the truth of AC is not touched, and Hilbert's First Problem remains unanswered. It is conceivable, even likely, that it will never be solved, despite Hilbert's optimistic slogan expressed in his Paris lecture: in mathematics there is no ignorabimus.
Pros and Cons of AC:
Herrlich's book is an interesting source of information around AC. He presents many equivalents of AC and some related concepts to AC. The main part are the chapters Disasters without Choice consisting of 11 sections organised by mathematical disciplines and the chapter Disaster with Choice consisting of 2 sections. To get a glimpse about such consequences I pick out two, three easy understandable examples:
From Section 4: Disasters without Choice
Section 4.4: Disasters in Algebra I: Vector Spaces
In ZFC every vector space is uniquely determined, up to isomorphism, by a single cardinal number, its dimension. Each of the two fundamental results which together enable us to associate dimension with a given vector space fail badly in ZF.
Disaster 4.42: The following can happen:
- Vector spaces may have no bases
- Vector spaces may have two bases with different cardinalities.
Theorem 4.44: Equivalent are:
- Every vector space has a basis
- AC
Section 4.6: Disasters in Elementary Analysis: The Reals and Continuity
Disaster 4.53: The following can happen
- $\mathbb{R}$ may fail to be Fréchet, i.e., not every accumulation point $x$ of a subset A may be reachable by a sequence $(a_n)$ in $A$.
... (9 more to follow)
Though the Axiom of Choice is responsible for many beautiful results, it is equally responsible for the existence of several dreadful monstrosities - unwelcome and unneeded.
From Section 5: Disasters with Choice
Section 5.1: Disasters in Elementary Analysis:
Definition 5.1: The equation $f(x+y)=f(x)+f(y)$ is called the Cauchy-equation
Consider a function $f:\mathbb{R}\rightarrow\mathbb{R}$ that satisfies the Cauchy-equation for all real $x$ and $y$. Then it is easily seen that
- $f(r\cdot x)=r\cdot f(x)$ for all rational $r$ and real $x$, i.e. $f$ is $\mathbb{Q}$-linear.
In particular:
- $f(r)=f(1)\cdot r$ for all rational $r$.
And continuity of $f$ would imply
- $f(x)=f(1)\cdot x$ for all $x\in \mathbb{R}$
Are there solutions of the Cauchy-equation that fail to be continuous? None has ever been constructed and in ZF none will ever be. However the Axiom of Choice guarantees the existence of such monsters; even worse, under AC there are far mor undesirable solutions of the Cauchy-equation than there are desirable ones:
Disaster 5.2: In ZFC there are
$2^{\aleph_0}$ continuous solutions $f:\mathbb{R}\rightarrow\mathbb{R}$ and
$2^{(2^{\aleph_0})}$ non-continuous solutions $f:\mathbb{R}\rightarrow\mathbb{R}$ of the Cauchy-equation.
Conclusion: It is good to be aware that there are benefits and drawbacks in ZF as well as ZFC and it's plausible that this is also the case for other set theories. So, there is no true definition of a set theory as framework for a true definition of a set.
Two hints:
A classic source to read and think about sets is Naive Set Theory by P. Halmos. It presumably covers most of the aspects around sets you might need for daily work.
On the other hand if you are curious, to see which functions in real analysis can be defined or not be defined according to underlying set theories, you may want to have a short look into Strange Functions in Real Analysis.