I started reading about Gödel's theorems recently and found the idea of using the tools of mathematics to understand logic and what we can and cannot do with it. While doing my problem set for a basic real analysis class--in which we've been building the reals up from Peano's axioms--I got to wondering what the chances are that interesting results come out of some set of axioms and definitions. Is it at all surprising that the particular axioms we assume to be true yield so many amazing results when combined with the right definitions? If so, what's so special about the number system we find ourselves with. OTOH if it's not surprising, then what makes such fruitful axiomatic systems so prevalent?
I doubt the question as I've posed it has any sort of answer, but I'm curious what sorts of questions related to this have been asked within mathematics. Where would I look if I wanted to learn about the tools for asking these sorts of questions?