How to properly define the category of sets? I'm reading a book on commutative algebra, where there is an appendix on category theory.
It defines a category $\mathcal{C}$ as having a set of objects $\text{Ob}(\mathcal{C})$ and a set of arrows between objects $\text{Hom}(\mathcal{C})$ satisfying conditions of composition, identity and associativity.
I'm a little troubled because I don't see how we can define the category of sets this way. 
Don't we have the classical paradox of "the set of all sets"?
What am I missing?
 A: We do not run into that "classical paradox", known as Russell's paradox, because we have the notion of class.  Class is a broader notion of collection than set; all sets are classes, but not all classes are sets.  Classes which are not sets are called proper classes.  In this framework, what Russell's paradox shows is that the class of all sets is a proper class, because if the class of all sets was a set we would get a contradiction.  Now you might be wondering, why can't we form a new Russell paradox about the class of all classes?  The answer is that there is no class of all classes, because not all classes are allowed to be the elements of a class.  Only a set can be an element of a class, a proper class cannot be an element of a class.  And how do you tell which classes are sets and which ones are proper classes, short of deriving a contradiction?  The approach mathematicians have settled on (though by no means the only consistent approach) is to look at its "size".  Proper classes are all of the same size; they can all be put in one to one correspondence with the class of all sets.  And if a class is too small to be put in one-to-one correspondence with the class of all sets, i.e. it has an actual cardinality, then we call it a set.  This is all formalized in set theories like NBG and MK.
So to answer your question, it's not a problem that all sets do not form a set.  Because the objects in a category just need to form a class, not a set.  In fact categories where there is a set of objects rather than a proper class of objects have a name, small categories.  And we can even create a category of all small categories, with functors as morphisms.  That's legitimate because small categories are sets, so we can form a class out of them.  But obviously the category of small categories is not a small category.
So to sum up, the category of sets is a perfectly legitimate category, it's just not a small category.
A: It is possible that your source is using a different foundational system than one involving classes. Using the notion of Grothendieck universe, it is possible to define categories so that the collections of objects and morphisms are always sets. The idea is to fix a strongly inaccessible cardinal $U$ and define the category of sets as the category of small sets, that is, those sets which are an element of $U$. The set $U$ is so big that this includes every set used in typical mathematics, as well as their (iterated) powersets and unions. 
This foundation is quite popular and common in areas in which category theory is commonly applied. To be fair, it has one weakness: ZFC set theory is strictly weaker than its augmentation with the axiom "there exists a strongly inaccessible cardinal $U$", whereas there is a formalization of the set-class theory called NBG set theory which has the same consistency strength as ZFC.
