Is Exercise 8.3 in Rudin's Principles of Analysis as easy as it seems? Rudin Theorem 8.3 says that if $$\sum_{j=1}^\infty |a_{ij}| = b_i$$ and $\sum b_i$ converges, then $$\sum_i \sum_j a_{ij} = \sum_j \sum_i a_{ij}$$
Rudin 8.3 asks us to show that if $a_{ij} \geq 0$ for all $i,j$, then $$\sum_i \sum_j a_{ij} = \sum_j \sum_i a_{ij}$$ including the case $\infty = \infty$.
It seems to me that the conditions of the Theorem follow pretty simply from the fact that $a_{ij} = |a_{ij}|$, and so we can say that if the LHS converges then the RHS converges, and to the same number, and that if the RHS converges, then the LHS converges, and to the same number. So the LHS converges if and only if the RHS converges to the same number. If one side diverges, then the other also diverges, and they both must diverge to $+\infty$. Hence we have equality.
But that seems too easy. Is there something I am missing?
 A: This is sometimes called the Fubini theorem for sums. If we had a double sum with finite indexes, the case would be trivial, i.e. just to transpose the table with i rows and j columns.
Yet, in our case we have a double series. Something of a different nature, the creature we are dealing with is some limit point in the complex plane with a key property that the way it is constructed permits us to flip the order of indices.
Warm-up
Imagine we have an infinite table with rows indexed with $i$ and columns indexed with $j$. By assumption, for this table the terms in each row add up to $b_i \in \mathcal{C}$, and this series for $b_i$, in its turn, converges to some complex number across all rows.
Here is how it looks.

We first sum up across all rows and get $b_i$ (blue arrow marked 'a'). Then across all $b_i$ down (violet arrow marked 'b'). We arrive at a certain sum, which in the original theorem essentially named $g(0)$.  
The proof
The proof basically explains why we can first move down (grey arrow 'c') and then right (red arrow 'd').
We can do this because the sum it each column j uniformly converges [to the limit function $g(x)$] by the Weierstrass theorem (Rudin 7.10 page 148). In the picture the limit function is the bottom red arrow and the nasty yellow band around it means that the sums across columns uniformly converge to this red arrow (row with index 'infinity').
Then we can apply either Rudin 7.11 or 7.12 to state that the limit function $g(x)$ (red arrow) is in fact continuous, including the limit point $g(0)$.
Another look
Up to now we have shown we can move both a -> b (right-bottom), c -> d (bottom-right). We only need to show that either way we arrive at the same limit sum. Here is the key bit.
$$ \sum_{i=1}^{\infty} \sum_{j=1}^{\infty} a_{ij} = \lim_{n \to \infty} \left( \sum_{i=1}^{\infty} \underbrace{ \sum_{j=1}^{n} a_{ij}}_{Comment [A]}     \right) = \lim_{n \to \infty} \left( \sum_{j=1}^{n} \underbrace{\sum_{i=1}^{\infty} a_{ij}}_{Comment [B]} \right) = $$ 
$$ \sum_{j=1}^{\infty} \sum_{i=1}^{\infty} a_{ij} $$
Comment [A]: this makes sense for all $i$ because across each row $i$ the sum converges (to $b_i$). On this side of equality this expression describes we first add up the terms in each column, and then sum across rows $i$, a -> b.
Comment [B]: this makes sense because for every column $j$ the sum of its terms converges uniformly to our limit function $g(x)$. On this side of equality this expression describes movement c -> d.
In the original proof $x$ essentially designate columns. As we sum across all columns we approach $g(0)$.
It is exactly for this reason we change from infinity to $\lim_{n \to \infty}$ for $j$ and not $i$! It is because we make use of the uniform convergence of the sums across all columns ($f_i(x)$ in the original proof)!
$$ \sum_{j=1}^{\infty} a_{ij} = \lim_{n \to \infty} \sum_{j=1}^{n} a_{ij} $$
A: @jme,
I don't think that it includes the case $\infty=\infty$.
Because the sum in LHS converges since $\sum b_i$ converges.
Next, if in a double summation there involves a finite sum, then the order of the summation can be swapped.
Indeed,
$$\sum_{i=1}^{\infty}\sum_{j=1}^{n}a_{ij}=\sum_{i=1}^{\infty}{(a_{i1}+\ldots+a_{in})}=\sum_{i=1}^{\infty}a_{i1}+\ldots+\sum_{i=1}^{\infty}a_{in}=\sum_{j=1}^{n}\sum_{i=1}^{\infty}a_{ij}.$$
Rudin just applied this fact look.

