This answer is very late, and I think others have answered quite adequately, but I do have some thoughts on this.
As others have pointed out, no one really cares what ordered pairs are, only what they do. You can imagine that a book on axiomatic set theory might include the following text:
"PROPOSITION. Given sets A and B, there exists a set A×B and two functions p₁ : A×B → A and p₂ : A×B → B such that, for any elements a ∈ A and b ∈ B, there is a unique element (a,b) ∈ A×B with p₁(a,b) = a and p₂(a,b) = b. (This set is a product of A and B, its elements are ordered pairs, and the functions are projections.)
"PROOF. Given a ∈ A and b ∈ B, let (a,b) := {{a},{a,b}}, where a ∈ A and b ∈ B; then ..."
Having proven this proposition, the author might later say, "Consider the subset of A×B given by...". But he probably wouldn't refer back to the proof, and in particular not to its use of the Kuratowski definition. All the important properties of ordered pairs are contained in the proposition itself.
What you might then ask is, "Why define ordered pairs as sets at all, rather than taking them as primitive?" The answer is that we simply have no need to do so. From an external perspective, every theorem of ZFC is actually a theorem about models of ZFC, saying in effect "Any structure that satisfies the axioms of ZFC must also satisfy this property." From this perspective, adding unnecessary axioms is equivalent to adding unnecessary hypotheses to a theorem: unaesthetic and possibly misleading.