Worrying about logic and foundations too much I am studying undergraduate physics and I am always interested in why we consider certain physical models instead of the others. That ultimately leads to the question why we consider one mathematical description over the other in a specific physics problem. 
Unfortunately these kinds of questions now make me struggle to learn anything new. In this question I would be happy to receive some advice or some personal experience. 
I do not know why but I have the philosophy that we do not understand physics because we do not understand mathematics good enough. Therefore, I took some mathematics courses to make the best of my physics education. I took, for example, set theory, which is assumed to be foundations of mathematics, but I had trouble understanding it as when we wrote something intuitively (such as index sets) then I had great trouble as I wanted for it to be defined well and not just "intuitive". I understood that even in mathematics a lot of proofs are based on our intuition probably assuming that good mathematician could make intuitive argument rigorous. I tried then to learn basics of logic to make my arguments more rigorous and then started spending too much time on understanding why we choose one definition over the other (for example, why we assume vacuous truth). In the end, no matter what subject I take, I get destroyed by the need to try to explain myself why we make definitions the way we do.
For example, the last thing I was thinking about was why do we use real numbers in the physics? For now, the most satisfactory answer would be that we measure some things using some apparatus, say, length with a ruler, and we divide it in equal parts to make it easier (say, in 10 parts). Then we measure length. But probably the object for which the length is being measured does not coincide with the marks on the ruler. So we divide each interval again in 10 parts. Then we make iteration (possibly infinite, whatever that means) until we obtain a number which would be infinite decimal by construction. And real numbers might be constructed as the set of all infinite decimals. 
Of course, I could assume that there exists set of real numbers with some properties that we want and then prove all the theorems, for example, in real analysis. But then why do we need exactly these properties?
Anyways, I have several possible answers on why this is happening :
1) I am not mathematically mature enough and this is something that stops being a problem with age, 
2) I am not doing mathematics right and this kind of behaviour just has to be stopped by myself because it is destructive (I spend too much time on it instead of studying things that I could apply and to do research on).
I would appreciate your comments!
 A: The use of the real numbers can be understood in some sense by their construction and in another by their utility:


*

*The natural numbers are just that, natural. If you want to describe the world around you, then counting is vital. Integers arise from that as you want some semblance of direction, i.e. if you give me a cat i have +1 cats, remove a cat and i have -1. The rationals then arise from the desire to partition things, e.g. I have 5 chocolate bars and six friends. Where, then, do the reals come from? This is a little more involved but if you take the rationals and then add in the limit points for every possible Cauchy sequence, you end up at the reals. There are all sorts of questions in nature where you want to find the limit of processes, so the reals end up being a very natural construction. The complex numbers come from algebraic closure but as regards real numbers, we use them because we want all of the above to hold and we also want an infinite field. There's also a historical basis, that moving from naturals->integers->rationals->reals->complexes is how human thinking has progressed.

*Utility. Physics does not determine absolute truth (i suspect expecting it to may be the root your philosphical dilemma). Nor is it supposed to. Physics constructs models of the world to allow accurate predictions to be made, nothing more. If i want to hit a boat with my cannon, i can use physics and be pretty confident it'll provide me the correct solution. If you want to heat a house, physics allows us to design systems that will do that within constraints of power, etc. Do we obtain the absolute truth of the universe? No. Are we always infallible? No. But physics is an incredibly useful tool. Universal truths are for philosophers to debate. Note that mathematics is not, in my opinion, universal truth either. We use ZFC for reasons of history and practicality. It is not the only option open to us.
A: There are parts of physics like quantum field theory that still have not been made rigorous, so if you insist on rigor in everything you do that will eventually be a serious problem in physics! Of course if you can reign in this impulse enough to get that far in physics, then perhaps your drive for rigor will one day help put that subject on firmer foundations.
Keep in mind that much of physics historically was done without being completely rigorous. Newton was doing calculus using differentials, yet he made it work quite well. In my experience the people who are best at physics are very good at straddling the line between intuition and rigor, and history shows that this ability was somewhat crucial in the development of physics. Imagine if we waited until all the mathematical foundations of calculus were in place before investigating classical mechanics using calculus. Physics would have been retarded by 2-3 centuries at least.
The people who are good at using math properly without using it rigorously are usually people who would make very good mathematicians, but they don't need the rigor to understand what is going on. Most physics students (again in my experience) are just mimicking this smaller group, and while they are certainly capable of computing, they don't entirely understand what they're doing, and often they don't realize that they don't understand what they're doing. Then there is a third group of people in physics for whom this kind of thing rubs them the wrong way or is too difficult, so they often drop physics and switch to math (like me!).
In summary, if you plan to stay in physics I think it is important that you learn to think intuitively about math even when you don't have rigor, and it can be a good exercise to see how much you can infer from the physics books before you open a math book to see how it should really be done. Developing this ability may help you one day in developing your own mathematical formulation of various physics.
Of course if this is too painful or difficult for you, as it was for me, then perhaps you are better suited for math.
A: What is the foundation of math?
Ultimately, logic. But getting from logic to set theory is much more difficult than it seems. Mathematicians more or less agreed on some self-evident truths they then call axioms (which kinda can be translated as "worthy" from Greek). You can almost measure the level of agreement on these axioms by counting the number of different axiomatic systems. 
If you want to, you can basically construct almost all mathematical objects by using the empty set $\{\}$ and the other axioms from ZFC (see here and here). There are some exceptions though (like categories and strongly inaccessible cardinals), but you can try to add axioms to add the exceptions to the mix (although they will stay incomplete).
Why are the definitions done the way they are?
Short answer: Because they work. Longer answer: They developed this way. Set theory alone has a rich history (ever wondered where $\in$/$\varepsilon$ comes from?), a short overview of it is given on Wikipedia. The development of set theory actually led to the question "What is the foundation of math?" (see first part of answer). But back to this question: It is kind of an evolutionary process, like all of science. What works well, survives. I managed myself to convince a group of students I tutored why it's good that linear maps between vector spaces are defined they way they are (because their kernel and image are then vector spaces again).
When you read scientific papers from the frontier of mathematical knowledge (e.g. on arxiv.org) you will often see new definitions which you may see never again or slightly changed if their current form turns out to have "flaws". But the mathematical basics, like almost anything done in the undergraduate courses, stand firm since about half a century and there is nothing to indicate that this would change soon.
Are you being too rigorous?
Well, that depends. Do you like to be rigorous? 
If not, then stop. Especially as a physics, you are allowed to do some handwaving. If you are simply ignorant of conditions like even being able to exchange limit and integral, you are to worry much less in most of your day-to-day physics, as I am told. The word "ignorant" here is used without prejudice, ignorance is bliss. I would fail at studying physics because I simply can't overlook these details and would invest too much time in it.
If you do like being rigorous, you may prefer doing more math, as others suggested. If usual math still takes to many intuition from you where you don't agree, I suggest working with theorem checkers, e.g. Mizar. If that still isn't rigorous enough for you, I don't know what to tell you anymore.
I have fun writing Mizar articles especially because it's so rigorous. Proving seemingly trivial things can take quite some time, because it isn't as trivial as it seems. If you want to learn more about Mizar, I recommend the PDF "Mizar in a Nutshell" or the PDF "Writing a Mizar article in nine easy steps". The official homepage is here, you may also like the HTML-linked articles, e.g. the one about boolean properties of sets (you can click on "proof").
This is a lemma I needed recently: Let $I$ be a set and $\{X_i\}_{i\in I}$ a family of sets (i.e. a function $X$ with domain $I$ and for all $i\in I$ holds $X_i$ is a set). Let $\prod_{i\in I} X_i$ denote the product set as usual (more formally it is the set of all functions $f$ with domain $I$ such that for all $i\in I$ holds $f(i)\in X_i$). Now fix $j,k\in I$, let A, B denote non empty sets and define the set families $\{Y_i\}_{i\in I}$, $\{Z_i\}_{i\in I}$ by
$$Y_i=\left\{\begin{array}{rl}
A&\text{if $i=j$}\\
X_i&\text{if $i\neq j$}
\end{array}\right.\qquad
Z_i=\left\{\begin{array}{rl}
B&\text{if $i=k$}\\
X_i&\text{if $i\neq k$}
\end{array}\right.$$
Now does $\prod_{i\in I}Y_i = \prod_{i\in I} Z_i$ imply $j=k$ and $A=B$? In fact not, as I found out, the trivial case $A=X_j, B=X_k$ is the exception (but it holds otherwise). And as "without loss of generality" as this may be, I have fun discovering these subtleties and proving them, rigorously :)
