If the iterated limits are equal does the double sequence converge?

If $(a_{m,n})$ is a double sequence (in $\mathbb{R}$ or $\mathbb{C}$) and $\lim_m\lim_n a_{m,n}=\lim_n\lim_m a_{m,n}=a$ then can we deduce $\lim_{m,n}a_{m,n}=a$? More specifically can we deduce $\lim_{n}a_{n,n}=a$?

I think this is true but I am not sure. My attempt at a rigorous proof:

Let $\epsilon>0$. Argument 1: $$\exists N_{\epsilon}\in \mathbb{N}: i\ge N_{\epsilon}\implies \left|\lim_ja_{i,j}-a\right|<\epsilon\text{ and }\left|\lim_ja_{j,i}-a\right|<\epsilon$$ This is the definition of the limit (I take $N$ as the maximum of the two $n_0$).

Argument 2: $$i\ge N_{\epsilon}\implies \lim_j\left|a_{i,j}-a\right|<\epsilon\text{ and }\lim_j\left|a_{j,i}-a\right|<\epsilon$$ This is an application of the limit laws.

Now fix $i\ge N$. Argument 3: $$\exists M_{\epsilon,i}\in \mathbb{N}:j\ge M_{\epsilon,i}\implies \left|a_{i,j}-a\right|<\epsilon\text{ and }\left|a_{j,i}-a\right|<\epsilon$$ This follows from the fact that if $\lim_n\left|b_n\right|<K$ then for large $K>0$, $\left|b_n\right|<K$.

Argument 4: As $\epsilon>0$ is arbitrary, $$\lim_j\left|a_{i,j}-a\right|=0\text{ and }\lim_j\left|a_{j,i}-a\right|=0$$ This is the wrong argment in my opinion as I have

$$(\forall \epsilon>0)(\exists N_{\epsilon}\in \mathbb{N}): \ i\ge N_{\epsilon}\implies (\exists M_{\epsilon,i}\in \mathbb{N})\ :j\ge M_{\epsilon,i}\implies \left|a_{i,j}-a\right|<\epsilon\text{ and }\left|a_{j,i}-a\right|<\epsilon$$ and from this I deduce $$(\forall \epsilon>0)(\exists M\in \mathbb{N})\ :j\ge M\implies \left|a_{i,j}-a\right|<\epsilon\text{ and }\left|a_{j,i}-a\right|<\epsilon$$ for large $i$ meaning

$$(\forall \epsilon>0)(\exists N,M\in \mathbb{N}): \ i\ge N_{\epsilon}\text{ and }j\ge M_{\epsilon,i}\implies \left|a_{i,j}-a\right|<\epsilon\text{ and }\left|a_{j,i}-a\right|<\epsilon$$

I basicaly lose the dependence on $i$. This is the difference for example between continuity and uniform continuity right?

So, $$i\ge N\implies \lim_ja_{i,j}=\lim_ja_{j,i}=a$$ (which I doubt). From here we let $i\to \infty$ and obtain the result.

-

Completely revised to match the expanded question:

Some extra notation will help me to discuss this. For $i\in\Bbb N$ let $r_i=\lim_ja_{i,j}$, and for $j\in\Bbb N$ let $c_j=\lim_ia_{i,j}$:

$$\begin{array}{cc} a_{0,0}&a_{0,1}&a_{0,2}&\dots&\to&r_0\\ a_{1,0}&a_{1,1}&a_{1,2}&\dots&\to&r_1\\ a_{2,0}&a_{2,1}&a_{2,2}&\dots&\to&r_2\\ \vdots&\vdots&\vdots&\ddots&\vdots&\vdots\\ \downarrow&\downarrow&\downarrow&\dots&\ddots&\vdots\\ c_0&c_1&c_2&\dots&\dots&? \end{array}$$

Your Argument 1 says that $|r_i-a|<\epsilon$ and $|c_i-a|<\epsilon$ when $i\ge N_\epsilon$. Argument 2 doesn’t say anything more. Now you fix an $i\ge N_\epsilon$. Your Argument 3 is incomplete. What you can argue is that there is an $M_{i,\epsilon}$ such that $|a_{i,j}-r_i|<\epsilon-|r_i-a|$ and $|a_{j,i}-c_i|<\epsilon-|c_i-a|$ whenever $j\ge M_{i,\epsilon}$, which then implies (via the triangle inequality) that $|a_{i,j}-r_i|<\epsilon$ and $|a_{j,i}-c_i|<\epsilon$ whenever $j\ge M_{i,\epsilon}$. However, the conclusion that you reached is correct.

As you thought, Argument 4 is the real problem, and for the reason that you give: you’ve lost the strong dependence on $i$ of $M_{i,\epsilon}$. I think that this dependence is even more evident in the expanded justification that I just gave for your Argument 3: you need to get $|a_{i,j}-r_i|$ not just less than $\epsilon$, but less than $\epsilon-|r_i-a|$, which could be a lot less than $\epsilon$ if $r_i$ is barely within $\epsilon$ of $a$. (And yes, this is similar to the difference between continuity and uniform continuity.)

-
Yes I understand the counter example. What I doubt is $\lim_ja_{i,j}=\lim_ja_{j,i}$ How is this possible? – Optional Apr 13 '13 at 6:59
@Optional: Why shouldn’t it be? I don’t see what you find odd. (And I wasn’t explaining the example: I was explaining why your argument doesn’t exclude it.) – Brian M. Scott Apr 13 '13 at 6:59
$\lim_ja_{i,j}$ is a sequence and $\lim_ja_{j,i}$ is another. Just because their limits are equal why should they be equal? – Optional Apr 13 '13 at 7:00
@Optional: $\lim_ja_{i,j}$ is a number, not a sequence. I don’t understand your confusion. Each column is a sequence, and each row is a sequence. It may happen that the row limits and the column limits are the same, as in Harald’s example. It may happen because the rows are identical to the columns, as in his example, or it may happen with rows that are different from the columns. – Brian M. Scott Apr 13 '13 at 7:04
@Optional: Oops; I just realized that I misread what you wrote. In fact your last step is wrong: the row and column limits don’t have to be $a$. Let me see exactly where you went wrong and revise my answer. – Brian M. Scott Apr 13 '13 at 7:11

No. Let $a_{m,n}=0$ if $m\ne n$, and $a_{n,n}=1$ for all $n$.

-
As I suspected. What is wrong in my proof? – Optional Apr 13 '13 at 6:24
I suggest you trace your proof using this specific example to find the mistake. (I won't be back for a couple hours. By then, either you will have found it, or someone else will have pointed it out.) – Harald Hanche-Olsen Apr 13 '13 at 6:32