Why does order matter in a conditionally converging series? Since elementary school, I was taught that addition is unaffected by order. Why does it matter in what order the terms of a conditionally converging series are added? Summation is addition, therefore shouldn't it have the same properties?
 A: 
Summation is addition, therefore shouldn't it have the same properties?

An infinite sum is not a sum in the usual sense. As it has been discussed early before, the operation of summation can only be considered for a finite amount of real numbers. Given a sequence $\{a_n\}$ of real or complex numbers, we can define a new sequence, closely associated to it by $$s_1=a_1$$ $$s_{n+1}=s_n+a_{n+1}$$
We're just inductively defining the sequence $a_1,a_1+a_2,a_1+a_2+a_3,\cdots$ and we usually write $$\sum_{k=1}^na_k=s_n$$
This new sequence has a well established order of terms, and, if the sequential limit $$\lim \; s_n=\sigma$$ exists, then we note this number by $$\sum_{n=1}^\infty a_k=\sigma$$ and call it the sum of the sequence $\{a_n\}$. By no means we're "adding" terms, but we're considering the limit of a sequence $\{s_n\}$. Moreover, $\displaystyle\sum_{n=1}^\infty a_k$ is convenient notation, but it is not addition of terms. If we reorder $\{a_n\}$ into a new sequence $\{a_{f(n)}\}$ there is no reason the new associated sequence $\{s_n^\prime\}$ will be the same as our original $\{s_n\}$.  As you're saying, we can reorder a conditionally convergent sequence to make it to whatver number $\alpha \in\Bbb R$ we want. Let's prove:
$\bf{LEMMA}$ Let $\{a_n\}$ be conditionally summable. Then $\{a_n^+\}$ and $\{a_n^-\}$ where $$a_n^+=\max(a_n,0)\\ a_n^-=-\min(a_n,0)$$ aren't summable. 
$\bf{PROOF}$ It is clear one must at least diverge, for if they both converged, we would have that $|a_n|$ converges. Indeed: $$0\leq \left| {{a_n}} \right| = \left| {\frac{{a_n^ +  - a_n^ - }}{2}} \right| \leqslant \frac{{a_n^ +  + a_n^ - }}{2}$$ so monotone convergence would imply $|a_n|$ is summable, which is contrary to our hypothesis. But in fact both must not be summable, and grow unboundedly large. If one was summable, and the other had bounded sums, then $\{a_n\}$ wouldn't be summable. 
Now, we can give a proof of Riemann's assertion.
THEOREM Let $\{a_n\}$ be conditionally summable. Then, for each $\alpha \in \Bbb R$ there exists a reordering such that $\{a_{p(n)}\}$ has sum $\alpha$, or either diverges to positive or negative unbounded values.
PROOF For easeness, let's consider $\alpha >0$. The lemma says $$P_n=\sum_{k=0}^na_k^+$$ and $$N_n=\sum_{k=0}^n(-a_k^-)$$ have unbounded positive and negative sums respectively. Let $n_0$ be the least integer such that $$\eqalign{
  & \sum\limits_{k = 0}^{{n_0} - 1} {a_k^ + }  \leqslant \alpha   \cr 
  & T_1=\sum\limits_{k = 0}^{{n_0}} {a_k^ +  > \alpha }  \cr} $$
In this case, we will have $$\eqalign{
  & a_{{n_0}}^ +  \geqslant \sum\limits_{k = 0}^{{n_0}} {a_k^ + }  - \alpha  > 0  \cr 
  & a_{{n_0}}^ +  \geqslant {P_{{n_0}}} - \alpha  > 0 \cr} $$
Let $n_1$ now be the least integer such that
$${T_2} = {P_{{n_0}}} + {N_{{n_1}}} < \alpha $$
We're simply adding some negative terms to get behind $\alpha$. Again, we'll have
$$0 < \alpha  - {T_2} <  - a_{{n_1}}^ - $$
We repeat this process to define inductively a new order of the terms of $a_n$ $$a_1^ + ,a_2^ + , \ldots ,a_{{n_0}}^ + ,a_1^ - , \ldots ,a_{{n_1}}^ - ,a_{{n_0} + 1}^ + , \ldots ,a_{{n_2}}^ + , \ldots $$
Note that since $\{a_n\}$ is summable, $\lim\;a_n=0$. But we constructed this reorder carfully so that in each step
$$|\Sigma-\alpha|<a_{n_k}$$
where $\Sigma$ will be some of the $T_n$ above, for some $k$ in this order. This means that for sufficiently large $n_k$, we will have 
$$|\Sigma-\alpha|<\epsilon$$ so indeed our sum will converge to $\alpha$.
A: It might be worth reading something like section 12-13 of Apostol's "Mathematical Analysis" or the rearrangements section of chapter 3 of Rudin's "Principles of Mathematical Analysis." Each quotes a result, and it seems to me it was due to Weierstrass, that if one has a series that is only conditionally convergent, and one choose two real numbers $x<y$ then there is a rearrangement of the series such that the sequence of partial sums of the rearrangement has lim inf = $x$ and lim sup = $y$. Choosing $x=y$ we can rearrange so that the new series converges to any real number. Behavior like this is proof that infinite sums need not behave the way we are used to finite sums behaving.
A: The supposition that infinite sums should behave like finite sums had led to many interesting, sometimes peculiar, results. Without getting into (potentially very much) detail I will just refer to the first chapter of "Adventures in Formalism", by Craig Smorynski for a very interesting historical account of various notions of infinite series that were entertained and studied by many mathematicians. 
To answer the question posed, a theorem due to Riemann states that given any conditionally convergent series and any extended real number (i.e., any real number, or infinity, or negative infinity) one can rearrange the given series so that the sum of the new series is the chosen extended real number. Moreover, one can find a rearrangement so that the new series does not converge at all. So, for conditionally convergent series, summation behaves quite differently than finite summation. In contrast, absolutely convergent series are much more well-behaved and do behave much more like ordinary finite sums. That is, for absolutely convergent series any rearrangement will result in a series that still converges and to the same sum.
A formal proof of Riemann's Theorem is not difficult at all. Here is a sketch of it. It is rather easy to show that a series is conditionally convergent if, and only if, its series of positive terms diverges to infinity and its series of negative terms diverges to negative infinity. Since the general term of a convergent series tends to $0$ one may view the positive terms as a source of magnitudes that get smaller and smaller but in such a way that any finite sum of a segment of those magnitudes can be made as big as one wants by choosing the segment to be large enough. Similarly for the negative part of the series. So, to rearrange the series to obtain a series whose sum is, say, $S$ simply take enough of the positive terms until you just get past $S$. Then add some of the first negative terms until you just get past $S$. Then add some more of the positive terms until you just get past $S$ again. Continue in this way forever to exhaust all elements in the series. 
Similarly, one can rearrange the series to converge to $\pm \infty $ or to not converge at all (nor diverge to $\pm \infty $).
A: A conceptually very simple example is the series
$1-1+1-1+1-1+...$.
If we group it as
$(1-1)+(1-1)+(1-1)+...$ it is $0$
since each term in parens is $0$.
If we group it as
$1+(-1+1)+(-1+1)+(-1+1)+...$ it is $1$
since each term in parens is $0$ so only the $1$ is left.
(I'm not sure this warrants being an answer, but what the heck.) 
