How to define mathematical objects from scratch?

Are there any "Zermelo-type theories" for some other type of mathematical objects except for sets? I.e., axiomatic systems for mathematical objects formalized for first-order logic using a language consisting of one or more non logical symbols (corresponding to $\in$)?

How to think about those systems, do they define the objects or the non logical symbols or both? Or are such systems merely statutes, rules for working with the objects and symbols? Could a system of this kind ever be categorical?

Edit: as the question is formulated Peano axioms for natural numbers and Tarski axioms for geometry are examples. Among others! But I thought I was asking for something else. But I think I learned something anyway, both from formulating the question and from reading the comments.

• Avoid such titles. If one searches for "scratch" in the web, one would not get a hit from this question. – Pedro Tamaroff Nov 8 '14 at 22:06
• Here are many examples of first-order theories: en.wikipedia.org/wiki/List_of_first-order_theories – Hanno Nov 8 '14 at 22:20
• @Arthur: As far as I know, Hilbert's axioms are not first order, but Tarski's are. – Hanno Nov 8 '14 at 22:31
• @Arthur - I do not thin so; see Hilbert's axioms : it a three-sorted anguage (points, lines, planes) but the Axiom of line completeness is stated in terms of "a system of points, straight lines, and planes". For an elementary axiomatization, see Tarski's axioms. – Mauro ALLEGRANZA Nov 8 '14 at 22:34