Is the notion of a set always a primitive notion? Is there an approach in set theory where we have a definition of a set, or do we always consider the notion of a set as being a primitive notion that cannot be defined in terms of previously defined other notions?
 A: The $\lambda$-calculus (and similar systems defined using combinators) can be seen as an attempt to define a mathematical world in which everything is a function. See https://www.encyclopediaofmath.org/index.php/Illative_combinatory_logic for references.
More recent work on mathematical foundation systems of this kind generally involves some notion of type (see http://en.wikipedia.org/wiki/Intuitionistic_type_theory) and it is a topic for debate whether type-theoretic foundations and set-theoretic foundations are really different. Proponents of type theory will generally say that they are.
A: There are candidates for foundational theories, such as category theory and the theories that Rob mentions, in which the notion of "set" is not primitive, but I don't think these count as set theories even if they turn out to be mutually interpretable with set theories.
In set theory you can get close to what you want, I think, with NBG set theory (the one-sorted formulation) in which "class" is the primitive notion and a set is defined as a class that is contained in some other class.
