Changing Order of Sums of Non-Negatives I want to prove something and I would like to use the identity: $$\sum\limits_{i=1}^\infty\sum\limits_{n\in A_i}a_n=\sum\limits_{n \in \cup_{i=1}^\infty A_i}a_n$$
Here $A_i$ are disjoint sets of natural numbers and $a_n$ are non-negative real numbers. It seems like the identity should work, but I can't find a good argument for why. It seems rather important that $a_n$ are all non-negative. My question is when can I justify changing the sums like this?
 A: *

*Changing the order of terms in a series can sometimes change whether and to what value it converges to.

*Some series converge to the same value no matter how you rearrange terms. These series are called unconditionally convergent. In symbols:
$$\sum_{n=1}^\infty a_n = \sum_{n=1}^\infty a_{\sigma(n)} < \infty$$
for any permutation of indices $\sigma$.

*It turns out that because the real numbers are complete, every absolutely convergent sequence of real numbers is unconditionally convergent. This means that if the sum of absolute values converges— i.e. $\sum_{n=1}^\infty |a_n|<\infty$, — then the series converges to the same value no matter how you rearrange terms.

*But a convergent series with no negative terms will always be absolutely convergent, and therefore unconditionally convergent.

*Next, note that sums like $\sum_{x\in X} p_x$ do not specify an order in which to add terms. As a result, they are not always well-defined (i.e. you can't tell whether they converge, or to what value.)
For example, suppose a series $\{a_n\}$ converges but not unconditionally, (e.g. if it has some negative numbers in it), and let $A$ be the (multi)set of values in $\{a_n\}$, allowing duplicates, then
$$\sum_{a\in A}a$$ 
is not well-defined because the answer you get, or whether you get a convergent value at all, depends on the order in which you add the values. And sets don't specify an order.

*A sum like $\sum_{a \in A} a$ has a well-defined value only if it doesn't matter in what order you add up the terms in $A$. In other words, only if every possible way of ordering them yields the same sum.

*For similar reasons, in your example $A \equiv \bigcup_{i=1}^\infty A_i$ is a set, and therefore the expression $$\sum_{n \in \bigcup_{i=1}^\infty A_i} a_n$$ only has a well-defined value if you get the same answer no matter how you iterate over the set $A$.

*But all of the numbers $a_n$ are positive. And the series converges when you add it up in the special order 
$$\sum_{i=1}^\infty \sum_{n\in A_i} a_n$$
therefore the series converges unconditionally. You can add up the $a_n$ in any order you want, and you can therefore express this as:
$$\sum_{i=1}^\infty \sum_{n\in A_i} a_n = \sum_{n \in \bigcup_{i=1}^\infty A_i} a_n$$
