Are there ways to build mathematics without axiomatizing? Every time I read about a theory in mathematics, it usually starts with axiomatizing the most fundamental concepts that are going to be treated.
Recently, I have started finding this troubling. In the foundational crisis, we tried to root all of mathematics on set theory and build it up from there. I believe this is a supremely elegant idea, but I have to ask myself why.
I understand why axioms are the brick wall against which all infinite regressions crash. We cannot, after all, ask why something is indefinitely. There must come a time when we simply say: because it is.
But why? What happens if we throw logic out of the window and attempt to start everything from scratch? 
I have read about model and category theory and all types of order logics, but none of them seem to be enough because they are all rooted in something that eventually leads to a so-called "self-evident truth". What if infinite regressions are similar to infinite series: something that at first we assumed was non sensical but actually turns out to be really useful?
My question is: are there ways to build mathematics without axiomatizing? If no, is there a proof?
 A: I remember reading the abstract of an article (or description of a book perhaps) that claimed to answer this using the principles of evolutionary biology; essentially, the author performed various simulations suggesting that organisms that take, as their fundamental logic, anything other than $2$-valued boolean logic tend to die off in the long run. I think if you Google around, you'll probably be able to dig something up in that vein.
One might object: ah, but you're using classical logic to build computer simulations and interpret the result of those simulations. That's circular! My gut feeling is that actually, this isn't circular (but my thoughts on this aren't sufficiently well-developed that its worth me trying to write them here.)
A: Gödels incompleteness theorems are widely interpreted to mean that it is impossible to construct a reasoning system that can prove itself to be consistent. As such, it is almost universally believed that it is impossible to construct mathemathics without starting with some "self evident" axioms.
The Wikipedia article explains the details of the theorems better than I can hope to do here and also includes sketches for various methods of proving the theorems.
A: Mathematics is about ideas (sometimes really great ideas) which must be formulated, because there are no visible mathematical objects in reality to rely on. We formulate these ideas using some human language enriched with mathematical symbols and definitions (also possible to express in human languages). 
The primary mathematical definition is 'mathematical statement'. Ordinary statements are often vague with a lot of implicit, unstated information attached to it and interpreted differently depending on personality and circumstances because reality is too complex, while mathematical statements about formal models is true or not true.
All mathematics is about formal models and all formal models is defined by axioms. Logic works when used intelligent, because it is consistent. Any consistent system will work for those who know how to use it.
Sitting at a table with the pieces of a puzzle, litterally och figuratively, it works to use a consistant system of classification: it brings order and progress. Logic includes rules to classify statements as true or not true in a consistant way.
A: We don't know that know that the logic we are using works.  There are an uncountable number of possible logical systems and there can't be a logical reason to select one logical system over another.
The decision to select a logical system is more the result of history and capacity to create results that are useful and pleasing to mathematicians.
