Question of double series convergence? As in this question, I am trying to understand convergence behavior of a double sum by numerical investigation.  The sum is
$$ \sum_{j=1}^\infty\sum_{k=1, k\neq j}^\infty \frac{1}{j^2-k^2}$$
Due to symmetry and cancellation, I get the result
$$\lim_{J \to \infty}\sum_{j=1}^J\sum_{k=1,k \neq j}^J \frac{1}{j^2-k^2}= 0$$
But if I evaluate $\sum_{j=1}^J\sum_{k=1, k \neq j}^K \frac{1}{j^2-k^2}$ and  increase $J$ and $K$ with $J \neq K$ there seems to be convergence to different values.  For example, if $K = 10 J$ then it appears to converge to $\approx 1.133$ and if $K = J^2$ then $\approx 1.216$.
What is happening here?
 A: It is not surprising that your numerical results are confusing.
First, be aware there is a distinction between the double series and an iterated series. When both sides converge it is true that
$$\sum_{j,k=1}^\infty a_{jk} = \sum_{j=1}^\infty \sum_{k=1}^\infty a_{jk}$$
where  convergence of the double series on the LHS means $\left|\sum_{j=1}^J\sum_{k=1}^Ka_{jk} - S\right| < \epsilon$ when $J,K > N(\epsilon)$ and convergence of the iterated series on the RHS means $S = \lim_{J \to \infty} \left(\lim_{K \to \infty}\sum_{j=1}^J\sum_{k=1}^Ka_{jk}\right) = S.$ 
When the terms can change sign, the double series can fail to converge even if iterated series converge. 
In this case, the double series is not absolutely convergent since 
$$\sum_{j=1}^J\sum_{k=1\\j \neq k}^K \frac1{|k^2-j^2|} > \sum_{j=1}^J\sum_{k=1\\j \neq k}^K \frac1{|k^2+j^2|} , $$
and the RHS diverges by the integral test. 
The series is also not conditionally convergent since it can be shown that for the iterated series,
$$\tag{*}\sum_{j=1}^\infty \sum_{k=1\\j\neq k}^\infty \frac{1}{k^2 - j^2} =  -\sum_{k=1}^\infty \sum_{j=1\\j \neq k}^\infty \frac{1}{k^2-j^2} = \frac{\pi^2}{8} \approx 1.2337$$
Taking $K = J^2$, your computations appear to be approaching this value.
When the iterated series have opposite signs, the double series can be convergent only if the value is $0$, which is not the case. The result (*) can be obtained using
$$\sum_{k=1\\j\neq k}^K \frac{1}{k^2-j^2} = \frac{1}{2j}\sum_{k=1\\j\neq k}^K \left(\frac{1}{k-j} - \frac{1}{k+j} \right) \\= \frac{1}{2j}\left(-\sum_{k=1}^{j-1}\frac{1}{k} + \sum_{k=1}^{K-j}\frac{1}{k} -  \sum_{k=j+1}^{K+j}\frac{1}{k} + \frac{1}{2j} \right)\\=  \frac{1}{2j}\left(\frac{1}{j} +\sum_{k=j+1}^{K-j}\frac{1}{k}  -  \sum_{k=j+1}^{K+j}\frac{1}{k} + \frac{1}{2j} \right)\\ = \frac{1}{2j}\left(\frac{3}{2j} - \sum_{k=-j+1}^{j} \frac{1}{K+k}\right) $$
and,
$$\sum_{k=1\\j\neq k}^\infty \frac{1}{k^2-j^2} = \lim_{K \to \infty}\frac{1}{2j}\left(\frac{3}{2j} - \sum_{k=-n+1}^{n} \frac{1}{K+k}\right)  = \frac{3}{4j^2}$$
Thus,
$$\sum_{j=1}^\infty \sum_{k=1\\j\neq k}^\infty \frac{1}{k^2 - j^2} = \sum_{j=1}^\infty \frac{3}{4j^2} = \frac{3}{4} \frac{\pi^2}{6}$$
