Every time I read about a theory in mathematics, it usually starts with axiomatizing the most fundamental concepts that are going to be treated.
Recently, I have started finding this troubling. In the foundational crisis, we tried to root all of mathematics on set theory and build it up from there. I believe this is a supremely elegant idea, but I have to ask myself why.
I understand why axioms are the brick wall against which all infinite regressions crash. We cannot, after all, ask why something is indefinitely. There must come a time when we simply say: because it is.
But why? What happens if we throw logic out of the window and attempt to start everything from scratch?
I have read about model and category theory and all types of order logics, but none of them seem to be enough because they are all rooted in something that eventually leads to a so-called "self-evident truth". What if infinite regressions are similar to infinite series: something that at first we assumed was non sensical but actually turns out to be really useful?
My question is: are there ways to build mathematics without axiomatizing? If no, is there a proof?