If CH can be neither proved nor disproved, and to assume it true or false can yield different results, isn't it an axiom?
The Continuum Hypothesis (CH) can be used as an axiom.
To elaborate on what you've stated in your question. I'll take Zermelo-Frankel set theory + the Axiom of Choice (ZFC) as the axiom scheme of standard mathematics. If standard mathematics (ZFC) is consistent, then so is ZFC+CH. Likewise, if ZFC is consistent, then so is ZFC+$\neg$CH.
So attaching it or its negation to ZFC does not change consistency.
But to be fair, any statement can be taken as an axiom -- even a false statement. The problem is that using a false statement (or inconsistent statements) as an axiom leads to a world where everything is both true and false (which isn't very interesting).
A more interesting question is, "Why isn't CH taken as an axiom (for standard mathematics)?"
One answer might be "Because we don't need it to do what we find interesting."
Another answer is "Because CH makes some cardinal arithmetic 'too easy'. $\neg$CH leads to more interesting mathematics."