set theory - sets containing something else In a book about logic (propositional logic), author uses sets to describe propositional variables, operators (eg. $A=\lbrace \neg,\wedge,\vee,\Rightarrow,\Leftrightarrow\rbrace$)... How can those sets even exist? What are those operators exactly? (obviously not sets, althought in axiomatic set theory everything is set, so does it means that sets such as $A$ does not exist? What is it then, just a proper class?)
So, in other words, are mathematical sets and sets in logic different? (Mathematical sets made by axioms, and sets in logic made just intuitively?)
Another thing which I would like to see is a step by step construction of all basic mathematics such that you start by nothing. In other words, no use of anything which hasn't yet been defined. (e.g., if you start with logic, then you cant use sets in logic, or functions, or numbers because you haven't yet even defined them). So, is there some book written in that way, or maybe some internet resource?
 A: First order logic may be encoded into ZFC, Zermelo Fraenkel set theory with the axiom of choice. 
One of the first constructions of set theory is the set $\omega$ of the natural numbers. To define $\omega$, we first let $0= \emptyset$. Then $1=\{\emptyset\}$, $2=\{0,1\}=\{\emptyset, \{\emptyset\}\}$, and so on. Formally, the construction is carried out solely using the axioms, and so while we are not starting with "nothing" we are starting with as little as possible. From here we go on to define the rational numbers, reals, relations, cartesian products, etc. 
After having constructed a sufficiently large amount of mathematics, we may begin encoding first order logic into ZFC. This is done using Godel numbering (see https://en.wikipedia.org/wiki/Gödel_numbering). Every logical symbol is assigned a natural number (and hence is associated with a pure set, since all the elements of $\omega$ are pure sets), and then doing first order logic within ZFC becomes a matter of arithmetic. 
Of course, the symbols $\neg, \wedge, \to$ etc are part of the language of set theory in the first place, and so you may feel that the problem of forming the set $A=\{\neg, \wedge, \to, ...\}$ persists in ZFC. The answer to this of course is that the collection $A$ can only be a set if by the symbols "$\neg$," "$\wedge$," "$\to$" you mean their encoding in ZFC as elements of $\omega$. 
But taken as symbols in the language of set theory, you are right, they are not sets, and hence do not exist from the perspective of ZFC. This is similar to saying that cows, lamps, and sandwiches cannot be sets, since they cannot be constructed using the axioms of ZFC.
For a good first book on formal ZFC set theory, see $\textit{Elements of Set Theory}$ by Herbert Enderton.
A: When you write about putting things like $\Rightarrow$ and $\neg$ into sets, they are from a formal perspective "just ink" -- in other words "$\Rightarrow$" and "$\neg$" are, for this purpose, nothing more than strangely shaped variable letters, chosen so to remind you what it is you intend to be doing with them.
You're then supposed to imagine that each of these symbols stand for some mathematical object that is appropriately chosen not to conflict with any other set you need to use during the argument you're carrying out.
If you take care not to use any particular properties of these objects, other than they're all different from each other and from everything that already has a meaning, you don't need to choose particular set-theoretic objects right when you start reading. Instead, when you're done reading and reach a theorem whose validity you need to convince yourself of, at that point you can choose some sets that don't like anything you have seen during the proof. This will always be possible, which is why we don't really care about actually choosing formal meanings for the symbols.
