Before the dam breaks, namely, the one that holds the waters of accusations, I want to specify that the question I'm asking is a "reference-request", and therefore does have an answer.
Often in mathematics one constructs a set of some sort, let's name it $A$. We've constructed it in an abstract way, so, a priori, structural aspects of $A$ are yet unknown to us, until we prove them. Lets say that, after some effort, we've proven that $A$ in fact has a set of properties $P$. If we happen to have previously studied, in general, the implications of the properties $P$, then we can now apply everything we studied about $P$ to $A$, and we all live happily ever after. It generally works this way, and people are considered satisfied with applying what we know about $P$ in $A$ if and only if we have proven that we can. Sometimes we can't, but in these cases there's an massive collective effort to do so.
In physics, is this the case? I'm very, very interested in finding out if anyone has made efforts to justify why we use mathematics to study the world outside of our minds. Because in doing so, we are treating the world as if it were $A$ in the example, and we are applying $P$, but without justification. Most likely there isn't such justification, because it'll all bubble down to some parmenides-style paradox about what-is-not, maybe... but I still would like to know about attempts, whether historical or modern.
Any books on the matter? online pdf's? I won't consider it an invalid answer if someone recommends some philosopher or some other, or literarian, or who-be-it.