Why does this series have a different sum when its terms are rearranged? The problem is:
Give an example of a convergent series such that, when the terms are rearranged, the series sums to a different value.
A solution is:

Although everything makes sense in this solution, I don't get how what it claims is possible.  How can a series have two different sums?  
Is this caused by the fact that subtraction is not commutative or associative?  Hence, in fact, we really have two different series here? 
 A: Recall that the sum of the series is the limit of partial sums.
Now, by reordering the sum, you change the partial sum, and hence you get a different sequence, which can converge to a different limit.
This is actually what happens with conditionally convergent series, but NOT with absolutely convergent ones. 
To understand better what happens, consider a series $\sum_n a_n$. Define
$$b_n=\left\{ \begin{array}{lr}
a_n & \mbox{ if  } a_n \geq 0 \\
0 & \mbox{ if  } a_n < 0 \\
\end{array}
\right. \\
c_n=\left\{ \begin{array}{lr}
a_n & \mbox{ if  } a_n < 0 \\
0 & \mbox{ if  } a_n \geq 0 \\
\end{array}
\right. \\$$
Then, if $\sum a_n$ is conditionally convergent, then $\sum b_n=\infty, \sum c_n = -\infty$. If we denote the partial sums by $S_, T_n$ and $R_n$ respectively then 
$$S_n=T_n-R_n$$
Now, by reordering, since both $b_n$ and $c_n$ take the zero value infinitely many times (again by conditinal convergence) you get new partial sums
$$S_n'=T_n'-R_n'$$
Since $\sum_{n} b_n =\infty$ you can make by rearangement $T'_n$ as big as you want. 
Also since $b_n=0$ infinitely many times, you can also make $T'n$ very small compated with $n$.
Same way, you can make $R_n'$ as big or as small as you want. Therefore, you can make the difference $S_n'$ anything you want
To make this more clear:
Pick your favorite number $a$. Rearrange the conditionally convergent series $a_n$ the following way: 
Start adding the positive terms ($b_n>0$) until the first time you exceed $a$. Then add negative terms until the first time you go below $a$. Then add the following positive terms until the first time you exceed $a$, then negative ....
Since $\sum b_n=\infty, \sum c_n = -\infty$, by the rearangement you get a series which oscillates around $a$ infinitely many times. Since, by the so called divergence test you get $\lim_n a_n=0$ the new series oscilates around 4a$ by bounds which converge to $0$, hence it is convergent to $a$.
But the number $a$ was chose arbitrary.
Again, the point is that by moving more positive or negative terms at the beginning of the series you get larger or smaller partial sums, which then can converge to a different limit. 
A: No, that's not the reason.
When we sum a series, the order by which we sum is essential. That is, we first take $a_1$, then $a_1+a_2$, then $a_1+a_2+a_3$ and so on. Then we see which limit we get.
If, instead, we sum it as$$a_1,a_1+a_3,a_1+a_3+a_2,a_1+a_3+a_2+a_5,a_1+a_3+a_2+a_5+a_7,a_1+a_3+a_2+a_5+a_7+a_4,\ldots$$then we may well get a different limit. However, if the series $|a_1|+|a_2|+|a_3|+\cdots$ converges, this cannot happen. That is, the limit will always be the same.
In order to know more, read about Riemann's rearrangement theorem.
A: This happens any time that you have an alternating convergent series which is not absolutely convergent. 
The interesting fact is that you can rearrange your series to converge to any number that you like. The reason is that the positive terms add up to $\infty$ and the negative terms add up to $-\infty$ so you can go as high or as low as you wish playing with positive and negative terms.
When you rearrange your series you get a different series so the limit is different.
A: I think that main (and only) reasons why sum of series that is not absolutely convergent is not invariant with respect to permutations are that the operation of subtraction is neither commutative nor associative in general.
Because, if the terms are all positive we obtain always the same sum, no matter how the series is rearranged, because of commutativity and associativity of addition over the positive numbers.
So, I tend to believe that you are on the right track when viewing different sums as a consequunces of non-commutativity (and non-associativity) of subtraction.
Because of non-commutativity and non-associativity of subraction it is natural to have different sums when we permute the series, but it can come as a surprise that we can obtain any value we want with different permutations.
