There is a lot of subtlety within the term "false statement". In the simplest sense, if we are studying a particular model or structure, each sentence in the appropriate formal language is true or false in that structure. There is little reason I can think to to try to pretend that a statement that is known to be false in a structure is true in that same structure.
But there are other senses of "true" and "false". For example, most mathematicians (if you give them no other context) would agree that "$2 + 2 = 6$" is false. If pressed, they might say they mean "false in the real numbers" - false in a particular structure. But, if we don't start talking about multiple structures, and we just talk in an informal way, most people who know basic arithmetic would say that "$2 + 2 = 6$" is false.
However, there are structures where $2 + 2 = 6$ is true. The simplest example is the finite field with two elements, $F_2$, in which $1 + 1 = 0$ and so $2 = 0$, and also $6 = 3 \times 2 = 3 \times 0 = 0$. So in this field, $2 + 2$ does equal $6$. On the other hand, we still have $0 \not = 1$ in this field - it is not true that all numbers must be equal just because we assume that $2$ and $6$ are equal. And there is a great use in studying finite fields like $F_2$ in many areas of mathematics. The key point is that $2 + 2 = 6$ is true in some other structure, not in the real numbers.
Similarly, there are other axioms which mathematicians, given no other context, would typically regard as "false". For example, most mathematicians accept there are Lebesgue nonmeasurable sets, which contradicts an axiom known as the Axiom of Determinacy. Only a vanishingly small number of mathematicians who know about the Axiom of Determinacy regard it as a "true" axiom, as far as I can tell. In fact, the Axiom of Determinacy is disprovable in ZFC set theory. But it is somewhat common for these same mathematicians to assume the Axiom of Determinacy in the study of descriptive set theory, because it has very beautiful consequences. People regularly enough publish peer reviewed papers which include theorems that assume the Axiom of Determinacy. One could say that these are examples of theorems proved from a "false" axiom.
In both cases (assuming $2 + 2 = 6$ in the context of fields, or assuming the Axiom of Determinacy in the context of set theory), we don't break mathematics. We just end up studying structures other than the usual ones.
In some cases, we can show there are no structures of a given kind that satisfy a particular axiom (e.g. there is no field with only one element). In that case, there would be little benefit in trying to assume the axiom. This situation can occur, for example, if we have already assumed other axioms that allow you to prove that a given axiom is false.
But, when some structures of a certain kind satisfy the axiom and others don't, just because we think the axiom is "false" in our favorite structure doesn't automatically make it uninteresting to study other structures where the axiom is true, provided that our other assumptions don't already prove the axiom is false.