Switching order of supremum for doubly indexed sequence? Suppose you have a doubly indexed sequence of reals, $(\alpha_{ij})$. Why is
$$
\sup_i \;\sup_j\ \alpha_{ij}=\sup_j\;\sup_i\ \alpha_{ij}?
$$
I know one approach is to note $\alpha_{mn}\leq\sup_j\;\sup_i\alpha_{ij}$ for any $m$ and $n$. Why is this exactly? I don't really know what $\sup_j\sup_i\alpha_{ij}$ means. Does it mean first fix some $j$ and find the supremum of $\alpha_{ij}$ as $j$ remains fixed as $i$ runs over $\mathbb{N}$? And then after that, find the supremum of $\sup_i\alpha_{ij}$ as $j$ runs over $\mathbb{N}$? It's not clear how I would find $\sup_i\alpha_{ij}$ for fixed arbitrary $j$ first. Thanks.
 A: To directly argue:
$$
a_{ij} \leq \sup_j a_{ij}\implies \sup_{i}a_{ij} \leq \sup_i\sup_j a_{ij} \implies
\sup_j\sup_{i}a_{ij} \leq \sup_i\sup_j a_{ij}
$$
A: Yes, it means exactly what you wrote.
The reason that
$$\sup_i\,\sup_j\ \alpha_{ij}=\sup_j\,\sup_i\ \alpha_{ij}$$
is that they're both equal to 
$$\sup_{i,j}\,\alpha_{ij}\;.$$
For assume that
$$\sup_i\,\sup_j\ \alpha_{ij}\lt\sup_{i,j}\,\alpha_{ij}\;.$$
Then $\sup_i\,\sup_j\ \alpha_{ij}$ is not an upper bound for the $\alpha_{ij}$ (since there is no upper bound less than the supremum). Thus there is some $\alpha_{kl}$ greater than $\sup_i\,\sup_j\ \alpha_{ij}$. But this $\alpha_{kl}$ would make $\sup_j\alpha_{kj}$ be at least $\alpha_{kl}$, and thus $\sup_i\,\sup_j\alpha_{ij}$ would also be at least $\alpha_{kl}$, a contradiction.
Similarly, if
$$\sup_i\,\sup_j\ \alpha_{ij}\gt\sup_{i,j}\,\alpha_{ij}\;,$$
then some $\alpha_{kl}$ would have to be greater than $\sup_{i,j}a_{ij}$, which is impossible.
Note that this only works because both operations are suprema. If you take, say, the infimum with respect to $i$ and the supremum with respect to $j$, then it does matter in which order you perform those operations.
A: Both are the same as 
$$\sup_{(i,j)\in\mathbb{N}\times\mathbb{N}} \alpha_{i,j}.$$
A: Suppose that $\lambda < \sup_i \sup_j \alpha_{i,j}$.  Then there must be some $i$ so
$\lambda < \sup_j \alpha_{i,j}$.  Hence there is some $j$ so that $\lambda <\alpha_{i,j}
\le \sup_{i,j} \alpha_{i,j}.$
We have $$\sup_i\, \sup_j \alpha_{i,j}\le \sup_{i,j}\alpha_{i,j}.$$  
Now suppose that $\lambda < \sup_{i,j} \alpha_{i,j}$.  Then there is some $(i,j)$ so
$\lambda < \alpha_{i,j}\le \sup_i \sup_j \alpha_{i,j}.$  Since $\lambda$ was chosen arbitrarily, the reverse inequality holds.
