Example of how set theory is foundation for the rest of mathematics I have heard it said that set theory can be seen as the foundation for the rest of mathematics. The article on set theory at brilliant.org put it as:

Set theory is important mainly because it serves as a foundation for the rest of mathematics -- it provides the axioms from which the rest of mathematics is built up.

I also saw an answer already on math.stackexchange.com that also further buttresses the point.
But the part I have failed to find is an example of such approach of using Set theory to formalize an other aspect of Mathematics.
For example, how would algebra or geometry or logic be expressed using set theory?
 A: To answer the last part of your question in detail ("how would algebra or geometry or logic be expressed using set theory?") would take way longer than a stackexchange answer. Very briefly, and in reverse order, and just for geometry:


*

*You can define the euclidean plane to be the set of all ordered pairs of real numbers. Similarly for 3-space.

*Real numbers can be defined as sets of rational numbers in various ways. For example, Dedekind cuts: for example, define the real number $\sqrt{2}$ to be the set $\{r \text{ rational} : r<0 \text{ or } r^2<2\}$.

*Rational numbers can be defined as sets of integers; for example, 1/2 is the set of all pairs of integers $(a,b)$ such that $b=2a$. (Think of a pair $(a,b)$ as representing $a/b$.)

*Integers can be defined using a similar trick: $-1$, for example, is the set of all pairs of natural numbers $(a,b)$ such that $b=a+1$. (Think of $(a,b)$ as representing $a-b$.)

*A pair of objects $(a,b)$ can be defined to be the set $\{\{a\},\{a,b\}\}$.

*Define the number 0 to be the empty set $\varnothing$. Define 1 to be the set $\{0\}$. Define 2 to be $\{0,1\}$. Etc.


If you want to see some of this fleshed out, the book by Enderton, Elements of Set Theory, is one reference.
I sketched how to grow a small part of the vast eco-system of modern math using just sets as the basic "biochemistry". Of course you'll also want to prove things about your mathematical objects, be they points, or vector spaces, or what have you. For this we need an axiom system for sets. Zermelo-Fraenkel set theory (ZFC) is a popular choice.
Now for the plot twist. This account of how set theory is "the foundation for mathematics" is kind of old-fashioned, and not everyone takes it that seriously any more.
It's not that there's anything wrong with the constructions (1)-(6). They're still perfectly valid. But if (say) you want to study euclidean geometry, only step (1) (also known as analytic geometry) helps you in any way. If you want to do real analysis, you can start by assuming the real numbers constitute a complete ordered field, and utterly ignore the possibility that a real number isn't just "a point", but has internal structure as (say) a Dedekind cut. Etc. And truth be told, some of the "encoding tricks" of (2)-(6) smack of contrivance, clever as they are.
Axiomatic set theory is a fascinating and beautiful subject in its own right. Also, the language of sets (and functions) has taken over math almost completely. Take something like Galois theory. If you look at pre-20th century treatments, the subject looks very different: all talk about rational expressions, permutations, and equations. Starting in the 20th century the vocabulary and conceptual framework changes: sets with structure, chiefly fields and automorphism groups. (Not that the rational expressions and permutations have disappeared!) It's a similar story for nearly every other branch of math, at least if the branch has pre-20th roots. Nowadays differential geometry is all about manifolds, but for Gauss it was about differential equations and surfaces in a more naive sense.
Lately (the past 40 years?) math has seen another framework shift, towards category theory. But that's another topic.
A: As an illustraton of what I intended to explain in this post, let me quote a comment (by a competent mathematician) that can be found on MSE regarding a recent question on the Axiom of Regularity : 
" It   is neither the case that “it should be true” nor is it the case that “it shouldn’t be true” [ "it" referring to the axiom of regularity] , any more than the Axiom of Choice “should be” or “shouldn’t be” true. It’s about what rules you want to play by. What you say about the natural numbers is, so far as I can tell, nonsensical. In ZF, everything is a set, period. Not just the natural numbers. They don’t have to be, but that is the most standard “rule” we play by, because it makes things simpler and achieves everything we need or want." 



*

*Maybe a  set theorist can be compared to a physicist.  


The physicist tries to explain observable processes regarding observable entities in terms of underlying unobservable entities governed by hypothetical laws logically organized. 
In the same way, in ordinary mathematics, we observe certain facts regarding given entities. Take, for example, the fact that every natural number has a succesor. 
The " ordinary" mathematician takes this fact at face value: for him, it is a " given". 
The set theorist asks,  the question : what is going on behind the scene when we go from a natural number $n$ to a natural number $n+1$? 


*

*At this point he makes hypotheses that he will consider as valuable in case they explain the observable process. 


He assumes he has an empty set ( $\{\emptyset\}$) and assigns to this set the role of number 0. 
He defines the observable process of adding 1 in terms of " successor relation " : 

assuming  numbers $n$ and $m$ are sets ( these are the unorservable
  entities)  then: 
$m=n+1$ ( observable fact)  $\iff m = n\cup \{n\}$ ( unobservable
  process).

( That is : the successor of a number is the union of this number and of the singleton having this number as unique element). 
He finally ends up with a set theoretic version of natural numbers : 

$\{\emptyset, \{\emptyset\}, \{\emptyset, \{\emptyset\}\}$....$\} = \{ 0, 1, 2, ...\}$

and with a set theoretic  definition of addition : 

$(1) n+0 =n $
$(2) n+S(m)=S(n+m) $

( Read " the sum of $n$ and of  the succesor of $m$ is the successor of $m+m$.) 


*

*I think the point of all this is the same as in physics : (1) explaining what we observe at the macroscopic level  ( relations between mathematical entities, operations, structures)  (2) in terms of more " fundamental" underlying entities  (3) through a minimal set of unifying consistent hypotheses (4) the consequences of which may allow an extension of our knowledge.  

