Is there any double series that cannot exchange the sum? I want to find a sequence $(u_{n,p})_{(n,p)\in\mathbb{N}^2}$ that satisfied:
$$
\sum_{n=0}^{\infty}\sum_{p=0}^{\infty}u_{n,p} ~\text{is convergent}
$$
$$
\sum_{p=0}^{\infty}\sum_{n=0}^{\infty}u_{n,p} ~\text{is convergent too}
$$
but
$$
\sum_{p=0}^{\infty}\sum_{n=0}^{\infty}u_{n,p} \neq \sum_{n=0}^{\infty}\sum_{p=0}^{\infty}u_{n,p}
$$
 A: Let  $u_{mn} = 1/(m^2 - n^2)$ if $m \neq n$ and $u_{mn} =0$ if $m = n.$
Note that
$$\frac{\pi^2}{12} = \sum_{n=1}^\infty \sum_{m=1, m \neq n}^\infty \frac{1}{m^2 - n^2} \neq \sum_{m=1}^\infty \sum_{n=1, n \neq m}^\infty \frac{1}{m^2 - n^2} = - \frac{\pi^2}{12}.$$
By anti-symmetry, the double sum changes sign with an interchange of indices.
To find the sum, use
$$\begin{align}\sum_{m=1, m \neq n}^\infty \frac{1}{m^2 - n^2} &= \lim_{M \to \infty} \frac{1}{2n} \sum_{m=1, m \neq n}^M \left(\frac{1}{m-n} - \frac{1}{m+n} \right) \\ &= \lim_{M \to \infty} \frac{1}{2n} \left(\sum_{k=1}^{M-n} \frac{1}{k} -  \sum_{k=1}^{n-1} \frac{1}{k} -  \sum_{k=n+1}^{M+n} \frac{1}{k}\right)\\ &= \lim_{M \to \infty} \frac{1}{2n} \left(\frac{1}{n}-  \frac{1}{M-n+1} - \ldots - \frac{1}{M+n} \right) \\ &= \frac{1}{2n^2} \end{align}$$
A: Let $$u_{n,p} = \begin{cases}
1 & \text{ if } n = p+1,\\
-1 & \text{ if } n = p,\\
0 & \text{ otherwise}.
\end{cases}$$
On the one hand, we have:
$$\begin{align}
\sum_{n=0}^{\infty}\sum_{p=0}^{\infty}u_{n,p} &= u_{0,0} + \sum_{n=1}^{\infty}(u_{n,n} + u_{n,n-1}) &\text{(all other terms are 0)}\\
& = -1 + \sum_{n=1}^{\infty}(-1 + 1) &\text{(evaluate the terms)}\\
& = -1
\end{align}$$
On the other hand, we have:
$$\begin{align}
\sum_{p=0}^{\infty}\sum_{n=0}^{\infty}u_{n,p}& = \sum_{p=0}^{\infty}(u_{p,p} + u_{p+1,p}) &\text{(all other terms are 0)}\\
&=  \sum_{p=0}^{\infty}(-1 + 1) &\text{(evaluate the terms)}\\
&= 0
\end{align}$$
A: 
Theorem: Let $\sum_{i=0}^\infty a_i$ be a conditionally convergent series. Then for every $x\in\mathbb{R}$ there exists a $\sigma$ in the permutation group of $\mathbb{N}$ such that $$\sum_{i=0}^\infty a_{\sigma(i)}=x$$
Put in words, this theorem says that given a conditionally convergent series, for every real number there is some reordering of the sum such that the reordered sum gives the real number.

We can use this theorem to construct examples of series of the sort that you are interested in. Let $\sum a_i=\sum(-1)^i/i$. The double summation gives an ordering on $\{u_{n,p}\}$, specifically the order that they will be summed in. You can picture this as a $\mathbb{N}\times\mathbb{N}$ grid, where each row corresponds to some fixed $p$ and each column corresponds to some fixed $n$. Now we wish to lay out our sequence $a_i$ in this ordering so that going rows by columns gives a different answer than going columns by rows. Doing this is straight forward and simply requires making each row and column a conditionally convergent series as there are no repeated terms which guarantees that reordering the sum will change the value.
