When reading about formal systems one is often warned that, even assuming the system is consistent, one can't be sure that any theorem proved in the system is actually true. For instance, if one can prove $\neg G$ where $G$ is the Gödel sentence in some formal system $S$, then that doesn't immediately make $S$ inconsistent. $G$ might not be provable despite the fact that that's exactly what $\neg G$ asserts. $S$ would have proven a falsehood, making it unsound, yet it might still be consistent.
What bugs me is: if one has arrived at a falsehood, doesn't that imply that either one of the axioms was false or that one of the rules of inference was invalid (capable of deducing a falsehood from truths)? But if either of those were the case that would make the interpretation under consideration no longer be an interpretation at all. And that would annul the charge that $S$ was unsound.
Take the following section from Torkel Franzén's Gödel's Theorem - An Incomplete Guide to its Use and Abuse:
There is a class of statements that are guaranteed to be true if provable in a consistent system $S$ incorporating some basic arithmetic. Suppose $A$ is a Goldbach-like statement [a statement similar to the Goldbach conjecture where, if it is false, a mechanically verifiable counterexample exists]. We can then observe that if $A$ is provable in such a system $S$, it is in fact true. For if $A$ is false, it is provable in $S$ that $A$ is false, since this can be shown by a computation applied to a counterexample, and so if $S$ is consistent, it cannot also be provable in $S$ that $A$ is true. Thus, for example, it is sufficient to know that Fermat's theorem is provable in ZFC and that ZFC is consistent to conclude that the theorem is true. But in the case of a statement that is not Goldbach-like, for example the twin prime conjecture, we cannot in general conclude anything about the truth or falsity of the conjecture if all we know is that it is provable, or disprovable, in some consistent theory incorporating basic arithmetic.
The incompleteness theorem gives us concrete examples of consistent theories that prove false theorems. This is most easily illustrated using the second incompleteness theorem. Given that ZFC is consistent, ZFC + "ZFC is inconsistent" is also consistent, since the consistency of ZFC is not provable in ZFC itself, but this theory disproves the true Goldbach-like statement "ZFC is consistent."
So according to Franzén ZFC + "ZFC is inconsistent" is unsound because it proves the falsehood "ZFC is inconsistent". But that's not how I see it. To me any structure where "ZFC is inconsistent" is a false statement can't serve as an interpretation of this system to begin with. After all, one of the axioms is false.
The above reminds me of Saccheri's famous "the hypothesis of the acute angle is absolutely false; because it is repugnant to the nature of straight lines". Non Euclidean geometry isn't unsound just because some of its theorems are false according to the straight line interpretation. By the same token ZFC + "ZFC is inconsistent" isn't unsound just because one of its theorems is false in the standard ZFC structure.
But then this raises the larger question how any formal system could ever be unsound. It would seem that any structure that might hope to serve as a witness for the systems unsoundness would immediately disqualify from being an interpretation at all.