How should one interpret an interval like $[2,1]$? Normally, when we're dealing with intervals $[a,b]$, it is at least implied that $a\leq b$. In this case, there are multiple equivalent ways to define what the interval actually is:


*

*$[a,b]=\{x\in\mathbb{R}\colon a\leq x\leq b\}$

*$[a,b]:=\operatorname{conv}(\{a,b\})$.


While the former definition is the one usually taught, the latter seems rather more elegant to me, though the convex-hull operator is itself defined by a more complicated set-builder notation. As both are equivalent, the distinction is unnecessary.
It gets interesting when we have $[a,b]$ with $a>b$. Usually, I would consider this simply wrong, but sometimes you have a general interval $[a,b]$ and then it can be tedious and contraproductive to always have to require $b\geq a$. What such an interval is comes out differently from both definitions:


*

*Always the empty set. This is the usual interpretation according to Wikipedia, but it's not necessarily very useful.

*The set with "correctly-ordered" borders $[b,a]$.


The latter result seems more useful to me, for instance we have always
$$\mu([a,b]) = \mu([b,a])$$
$$
  \int\limits_a^b\mathrm{d}x\ f(x) = \int_{[a,b]}\!\!\!\mathrm{d}x\ f(x)\cdot\operatorname{sgn}(b-a)
$$
and so on.
What reasons are there to not just always define intervals this way?
 A: Because of the usual left to right reading order in European languages, $[a,b]$ has a useful connotation of order. However, your suggestion certainly has merit.  If it were the current convention, there would be no strong arguments for change. 
However, it is not the current convention.  In elementary mathematics, changing the convention would instantly make most books very slightly obsolete.  That might make the idea attractive to publishers, but surely not to anyone else. 
History has left us with quite a number of unfortunate notations.  If one introduces a new notation, or a new interpretation of a notation, ordinarily one of the following happens: (i) The new notation is not widely adopted, and quickly dies or (ii) The new notation or interpretation is attractive to many, and is adopted by many.  In case (ii), we end up with two notations or interpretations.
A: I can think of two reasonable interpretations. If you think of $[a, b]$ as defining the set $\{ x : a \le x \le b \}$ then clearly this is empty if $a > b$. This would be an appropriate convention in the context of general posets, where sums over all intervals in a poset should be interpreted like this.
The second is to regard $[a, b]$ as being the negative of $[b, a]$. That is, identify $[b, a]$ with its indicator function: then $[a, b]$ should be the negative of the indicator function. This would be an appropriate convention for defining signed integrals over intervals, which is really a special case of a contour integral. 
