I understand the concept of a set having closure under a particular operation; that, for example, the natural numbers are not closed under subtraction. But why does closure matter? Is it necessary for an operation to be closed to be useful? If the set is not closed under an operation does that mean that the outcome of the operation is not defined? Or is it only a warning not to assume the outcome of the operation to be the same kind of number?
The term "closure" is useful for communicating an idea to someone else. If you are discussing an operation on an underlying set, you might want to say "If I take two elements from my set and operate on them, will I end up in my set." But this is wordy, so instead you say "Is the set closed under this operation?"
The context will determine if it "matters." For example in Group Theory, we care about something called "Groups." But in order to have a group (which is a set and operation following particular rules), and all the nice properties that come with them, we need to have our operation be closed.
The outcome of the operation can still be defined. In the example you gave, 3-5 is certainly defined (because you get an answer), but the natural numbers are not, as you said, closed under subtraction, because you take two elements from your set, operate on them, and do not remain inside your set.
When we talk about a set being closed under a particular operation, we are indeed talking about whether or not the operation is well-defined and will yield the same kind of thing. It's possible that the operation has a perfectly well-defined rule, but gives us a different kind of thing (subtraction of natural numbers, for example), or it may be that the operation isn't even well-defined (such as multiplicative inverse on the set of all $2\times 2$ real matrices).