For the physics case, and I say this as a physicist not a mathematician having a dig:
Almost all of physics is "not strictly correct".
What I really mean is: "strictly correct" in mathematics means an entirely different thing to what it means in physics. In fact, in physics there is a much larger set of interpretations of the term than there is in mathematics. Mathematics as a discipline considers "correctness" as fundamental. Not so in physics.
People have different views as to how Physics works. Mostly it is about modelling the world such that experiments can be constructed that will show results consistent or inconsistent with the model.
What this means is that the mathematical correctness of the model is a completely different thing from its physical correctness.
Indeed, if you had a model known to be a bit flaky mathematically but able to be used to give experimentally valid results, then it is better than a total mathematically consistent one which does not.
Now you might say how can a "flaky" model not necessarily have a space of solutions that are non-physical and hence not experimentally valid. The answer is that those conditions may only appear in a situation we cannot actually observe or construct due to practical reasons.
So to come back to the claim almost all physics is "not strictly correct": there simply is not basis for ever saying that a theory is strictly correct. For example, the Lorentz velocity addition formula is "better" than the Newtonian one, but is a the end of the day a model applicable to specific situations only. We cannot for example, just go about adding velocities for bodies at different locations once we allow for General Relativity.
It is in this sense that the current state of the art of physics "is a good way to understand a concept until one gets to a more advanced stage". Only if and when we have a serious candidate for the Theory Of Everything can (maybe) we go beyond this.
So to cap my answer, I'd suggest excluding physics from the question such that this becomes a non-answer . . .