What is correct order of foundational concepts of mathematics? I am trying to understand the hiearchy of foundational concepts in mathematics/logic.  
Specifically, is it true that objects that are left undefined or primitive or atomic are in some sense "below" axioms?  As an example, in Euclid's geometry a point is primitive (undefined, at least mathematically precisely) whereas the first postulate (axiom) talks about how to define a line segment from two points.
Halmos' Naive Set Theory (p2) seems to get the "natural" picture above backwards since it makes a point of using an axiom regarding the equality of sets as more fundamental (implying) membership in a set which it previously referred to as a primitive concept in axiomatic set theory.
 A: Are you familiar with formal theories and models? It sounds like you are mixing these two different things. In fact, it sounds like you might be asking about mathematicians rather than about mathematical objects. The word "concepts" is a clue to this. Also, if you are thinking "first this happens, then that happens" (e.g., "first, you get some axioms, then you apply some rules"), you are probably thinking about mathematicians rather than mathematical objects. Anyway, I don't have an answer about mathematicians. But as far as your question applies to mathematical objects, the objects in a formal language are symbols, strings, terms, formulas, proofs, etc. Languages don't have points, lines, numbers, sets, membership relations, additions, etc. Those are parts of models. So it's not clear what it means to say that, say, a number is below an axiom that talks about numbers because the two things are not even in the same structure.
Let's look at the language first. We usually think of symbols as atomic. Then a formula (axioms and theorems are formulas) that can be interpreted to talk about numbers is just a bunch of symbols concatenated together. A (formal) proof is also just a bunch of symbols concatenated together. Or instead of concatenation, it's probably better to think of sequences, i.e., functions from indexing sets to the set of symbols. It's the same deal: everything is just a bunch of symbols in some order. Symbols are then below axioms in the sense that axioms are composed of symbols. 
But you could also think of the formulas as atomic. I've never seen this done, but you can just reverse the previous thinking. Instead of thinking of a formula as a function from indices to symbols (telling you which symbol is in each position), think of a symbol as a function from formulas to sets of indices (telling you in which positions (if any) the symbol occurs). I don't know why this way of thinking would be useful, but who knows. The point is that now axioms are below symbols in the sense that in order to identify a symbol, you have to look at the positions in which it occurs in which formulas. When symbols were below axioms, it was in the same sense: to identify an axiom, you had to look at which symbols occurred in its positions. You're looking at the same mapping in each case, just in a different direction. (The situation is only slightly different because you've singled out axioms, which aren't special with respect to this mapping. This is like singling out some type of symbol, say, binary predicate symbols. You can't identify axioms (as functions) by looking only at binary predicate symbols (as atoms). In the same way, you can't identify symbols (as functions) by looking only at axioms (as atoms). You need to consider all formulas and all symbols.)
When you interpret the language in some model, some of the symbols get mapped to numbers (and relations and operations on numbers), and the formulas get mapped to truth-values. It's not clear to me how numbers or any other objects in the domain of an interpretation are related to truth-values. Truth-values only need to have very simple properties -- possibly, they just need to be distinct from each other. The numbers and truth-values are related by a function that takes numbers (and their relations and operations), plugs them into formulas, and outputs truth-values. But you can think of this function as working in other direction also.
Also, since you are on axioms, note that there's nothing special about an axiom in that you can take any formula as an axiom. Axiomatizations are special. They are special because not every theory has axiomatizations with nice or decent properties.
I want to pretend to be old and wise and advise you to give up looking for "the correct order" because I think it's all relative. A lot of math (maybe all?) is about mappings, and you can often (maybe always?) reverse them in some way. But I think there are some interesting questions related to your question. They are questions about what can be defined in terms of what, when axioms are independent of each other, questions about properties of axiomatizations, and so on.
Also, the things that you might think to call "undefined" -- numbers in arithmetic, sets in set theory, etc. -- are not undefined at all. The whole theory defines them. Set theorists don't walk around with no idea of what a set is.
A: In logic, axioms are not necessarily "below" everything.  This, I think, becomes clear when you consider natural deduction systems with no axioms.  Do such systems stand on nothing?  No, they still have formation rules for formulas (wffs) and rules of inference which they "stand on".  So, I would answer that "yes, objects which get left undefined or primitive or atomic are in some sense "below" axioms."
Classical set theory presupposes classical logic.  One can see this if one tries to thoroughly prove something like the commutativity of union or intersection of sets, where, at least the proofs I've seen, end up referring to the logical properties of disjunction and conjunction respectively one way or another.  You might also find it to note that if you don't have classical logic presupposed, and say have an infinite-valued valued logic instead, you can end up with a fuzzy set theory instead of classical set theory.  The axiom of exetensionality (or axiom of extension) which you refer to presupposes that the notion of membership in a set only happens in the setting of classical logic (that is, objects either belong or do not belong to a set).  Membership does qualify as a primitive concept of classical set theory... more primitive than the axioms of classical set theory.  If we allow objects to partially belong to a set to varying degrees, then the axiom of extensionality fails as it does in fuzzy set theory.  I don't know what Halmos wrote exactly though, and you might want to check and see that you haven't misinterpreted him (you might not have, I only scanned that book many years ago).
