What makes a result in mathematics true is different from the manner in which an answer in physics is true. In mathematics, a lot of the time, we often sit here just messing with abstract notions that almost certainly have no analogous representation in reality, or at least an experimentally-verifiable one.
How, then, do we know if a result in mathematics is true?
It follows from the axioms of the mathematical framework in which we are working. (I mean, yes, if the context is appropriate, there are other ways in which to check your answer/logic/whatever it is may be. But I'm talking about the more "fundamental" notion of correctness, since this also has to apply to those methods of checking your answer.)
This is a powerful thought, because of two things. For one, it means that anything either is or is not true - there is not some notion of "well, close enough" or "kinda true" or whatever. The laws of physics have to be revised to account for new observations and is dynamic in that sense - some observations might be true, but not for the reasons we think - whereas mathematics is comparatively static: given a proper set of axioms, something either does or does not follow from it. The only thing is to determine if, indeed, it does follow.
(Of course, depending on the axioms involved, something might be literally unprovable. Godel's incompleteness theorem might be worth looking at. I believe one example of something that's unprovable would be the continuum hypothesis, at least in the framework of ZFC set theory.)
What I find more interesting is that it doesn't specify the axioms or mathematical framework in itself.
This is powerful in that we can take "obviously false" results from our current understanding, plop them in a different framework or axiomatic system, and see what comes of it. You could make the argument that, in a sense, this is where complex numbers came from. We saw square roots of negatives, decided they didn't exist, until we decided to investigate what happened if they did exist. And hell, who would've thought that from a simple notion that the field of complex analysis would be born - that it would even have applications outside of mathematics? These "obviously false" results could always be worked into a system in some way: if nothing else, we can discuss what comes of them. Who knows, they might even be useful!
So in this sense, almost everything in mathematics is actually true, if placed in the right context. It might not be true in this framework, but it might be true in this framework. Physics is sometimes concerned with similar notions - alternate universes, alternate laws of physics, etc. - but their ideas can be at best difficult if not impossible to verify experimentally (and thus be "right" in the physics sense). In mathematics, there's an unparalleled freedom since we can pick and choose our "universe" by playing with the axioms as we so choose.
Of course, this almost begs another question - why do we often favor the axiomatizations we do? After all, we have a literal sense of freedom in this respect, so why favor them?
That's a broader question I'm even less qualified to answer, but I believe it comes from two things:
They just make sense and are thus somewhat useful. I feel like there's a sort of underlying notion of intuition in mathematics, or that we at least seek this intuition to what degree we can. If we deliberately make systems that are hard to wrap our minds around in even an intuitive sense, not much is going to come from it, and certainly few applications. It's sort of like coding in that Brainfuck computer language: sure, you can, but why would you want to do that to yourself, and what could you realistically do with it that wouldn't be way easier with Python, Java, C++, or any number of other languages?
There's a "richness," as another answerer, ConMan, put it. There's a lot of interesting results that ultimately follow from the axiomatizations that we favor. An axiomatization that results in a lot of results bears further examination in its own right, particularly if said results are interesting or useful.
This also limits us somewhat. Okay, we can axiomatize however we want, yet we want axiomatizations with these certain, nice properties - intuition (well, maybe, I for one at least highly value it), richness, and the ability to be applied usefully are probably the big main properties we want, ignoring the obvious "we don't want the axiomatization to be self-contradicting."
So, in short:
- A result in mathematics is true if it logically follows from the axiomatization of your framework.
- What that axiomatization or framework is: that's effectively up to you, within a certain extent. There are results to be had from tweaking the axioms or framework a little to allow results that aren't normally worth considering.
- That said we favor certain axiomatizations for the reason that they're useful, that they work, and they're relatively easy to grasp. (And not self-contradicting hopefully.)
- Physics, as opposed to this, tends to rely more heavily on experimentally-verifiable results, which can be limiting in certain contexts, and also tends to have a dynamic set of laws and theorems that change as new observations come to light. This results in a fundamental groundedness in reality and experimentation that mathematics need not always have.