What's the definition of limit of sets(esp. ordinals) in set theory? By definition of exponent operator on ordinals, we have
$$0^\omega=\lim_{\xi\to\omega}0^\xi$$
However, Note that $0^\xi$ is not increasing, so if we still let $\lim_{\xi\to\omega}0^\xi=\sup\{0^\xi|\xi<\omega\}$ then it is followed by $$0^\omega=\sup\{1,0,0,\ldots\}=1$$ 
An incredible result.
Nevertheless if we use the definition of limit superior $$\overline{\lim}_{\eta\to\beta}\alpha^\eta:=\inf_{\eta < \beta}\sup_{\eta \le\xi<\beta}\alpha^\xi$$
and limit inferior $$\underline{\lim}_{\eta\to\beta}\alpha^\eta:=\sup_{\eta < \beta}\inf_{\eta \le\xi<\beta}\alpha^\xi$$
then we get $$\lim_{\xi\to\omega}0^\xi=\underline{\lim}_{\xi\to\omega}0^\xi=\overline{\lim}_{\xi\to\omega}0^\xi=0$$
Conceivable, but this is not the end of the story. Let's consider about $m^n$ with $0<m,n< \omega$. Since exponent operator is continuous in the second slot, this sentence following must hold: $$m^n=\lim_{\xi\to n}m^\xi=\underline{\lim}_{\xi\to n}m^\xi=\overline{\lim}_{\xi\to n}m^\xi=m^{n-1}$$ fail when $1<m$.
So can we find a perfect definition of limit on ordinals?
Update:
By the way, if we deal the Ordinal class as a discrete topology space, then $\{\alpha\}$ is a neighborhood of $\alpha$, hence $\{\alpha\}$ must in every filterbase which converge to $\alpha$. So if we let $$\overline{\lim}_{\eta\to\beta}\alpha^\eta:=\inf_{\eta \le \beta}\sup_{\eta \le\xi\le\beta}\alpha^\xi$$
and $$\underline{\lim}_{\eta\to\beta}\alpha^\eta:=\sup_{\eta \le \beta}\inf_{\eta \le\xi\le\beta}\alpha^\xi$$
Then $$\alpha^\beta=\lim_{\xi\to \beta}\alpha^\xi=\underline{\lim}_{\xi\to \beta}\alpha^\xi=\overline{\lim}_{\xi\to \beta}\alpha^\xi$$ hold for every $(\alpha,\beta) \in \mathbb O^2$
But actually it cannot be a definition since $$\inf_{\eta \le \beta}\sup_{\eta \le\xi\le\beta}\alpha^\xi:=\inf\{\sup\{\alpha^\xi|\eta \le\xi\le\beta\}|\eta \le \beta\}$$ but $\{\alpha^\xi|\eta \le\xi\le\beta\}$ contains $\alpha^\beta$!
Update: Edited title.
The eventual question is as title illustrates. 
 A: You asked in your comment:

Indeed, but what's the exact definition of ordinal limit?

Let me try to address this, although this is not an answer to your original question.

The notion of limit of of transfinite sequence is defined only for limit ordinals
see e.g. Wikipedia

If $\alpha$ is a limit ordinal and $X$ is a set, an $\alpha$-indexed sequence of elements of $X$ is a function from $\alpha$ to $X$.  This concept, a transfinite sequence or ordinal-indexed sequence, is a generalization of the concept of a sequence. An ordinary sequence corresponds to the case $\alpha=\omega$.

Some authors consider only increasing sequences, see e.g. Kechris: Classical Descriptive Set Theory p.349
or Sierpinski: Cardinal and ordinal numbers, p.287
or Kuratowski, Mostowski: Set Theory: p.231.
In the case of increasing (non-decreasing) sequence we have $\lim_{\xi\to\alpha} \beta_\xi=\sup\{\beta_\xi; \xi<\alpha\}$.

However, it makes sense to define limit of any transfinite sequence $(\beta_\xi)_{\xi < \alpha}$, if $\alpha$ is a limit ordinal. (We do not have to use only monotone sequence.)
A transfinite sequence transfinite sequence $(\beta_\xi)_{\xi < \alpha}$ converges
to an ordinal $\beta\ne0$ if, for every ordinal number
$\gamma<\beta$, there exists an ordinal number
$\eta < \alpha$ such that  $\gamma <\beta_\xi\le\beta$
whenever $\eta <\xi < \alpha$.
$$(\forall \gamma <\beta)  \quad
(\exists \eta < \alpha)  \quad
(\forall \xi) \qquad (\eta<\xi<\alpha
\Rightarrow \gamma < \beta_\xi \le \beta) $$
A transfinite sequence $(\beta_\xi)_{\xi < \alpha}$ converges to $0$ if it is eventually equal to zero.
$$(\exists \eta < \alpha)  \quad (\forall \xi) \qquad 
(\eta<\xi<\alpha \Rightarrow  \beta_\xi =0) $$

If you are familiar with nets, you can notice that this is the same as limit of the transfinite sequence $(\beta_\xi)_{\xi<
\alpha}$ considered as a net in the order topology on  $\lambda$, where $\lambda$ is any ordinal larger than all $\beta_\xi$'s.
The basis for the order topology on some totally ordered set $X$ consists of sets of the form $(-\infty,b)=\{x\in X; x<b\}$, $(a,\infty)=\{x\in X; a<x\}$ and $(a,b)=\{x\in X; a<x<b\}$ for $a<b$. 
It is not difficult to notice, that if $\beta>0$ then neighborhood basis of $\beta$ (as an element of some $\lambda>\beta$ with order topology) consists of all sets of the form $(\gamma,\beta+1)$ where $\gamma>\beta$. Precisely these sets were used in the above definition.
Neighborhood basis at $0$, cannot be described in that way, that's why I had to treat the case $\beta=0$ separately. (In this case, we can take the neighborhood basis consisting of the single set $(-\infty,1)=\{0\}$.) Thanks for letting me know that the definition I suggested originally wasn't working.
A: The answer $0^\omega = 0$ is correct, and intended.  The "sup" definition is a mistake someone made by overlooking this single case.
The "limit" definition for ordinals must be $\lim_{\xi \to n} f(\xi) = f(n)$ in case $n$ is not a limit ordinal.  This is the same as we use in any topological space, the limit at an isolated point.
A: The problem is that the ordinal topology is not the discrete topology. It is the order topology, namely an open set is an interval.
It is easy to see that successor ordinals are indeed isolated points, but limit ordinals are exactly those ordinals which are not isolated.
The definition of the exponentiation is somewhat topological and therefore when considering a limit ordinal we consider this a limit in the topological sense.
What may confuse you is that if $f\colon\mathrm{Ord\to Ord}$ then $\lim\limits_{\xi\uparrow\delta}f(\xi)=\sup\{f(\xi)\mid\xi<\delta\}$ if and only if $f$ is non-decreasing.
As you noted $0^0=1$ but $0^1=0$, therefore the function is decreasing and we cannot interchange the $\sup$ and $\lim$.
A: If $\beta$ isn't a limit ordinal, we can't really talk about limits of functions $f(\gamma)$ as $\gamma$ approaches $\beta$. After all, $\beta$ is isolated, and so we can't approach it arbitrarily closely.
If we are dealing with a non-decreasing sequence of ordinals, there are two possibilities: (1) The sequence is eventually constant, so has a greatest element, which will be the limit of the sequence. (2) The sequence is not eventually constant, so has no greatest element, so its supremum (and limit) will be a limit ordinal, specifically the least limit ordinal greater than every ordinal in the sequence.
If we are dealing with non-increasing sequences, they must be eventually constant, because every non-$\emptyset$ set of ordinals has a least member. This really precludes any kind of "convergence from above", so we will always be dealing with convergence from below (as Martin expresses explicitly in his answer).
Ordinal exponentiation is more accurately recursively defined as follows:
$\alpha^0=1$; $\alpha^{\beta+1}=\alpha^\beta\cdot\alpha$; and for limit $\lambda$, $\alpha^\lambda=\sup\{\alpha^\beta:0<\beta<\lambda\}$.
We exclude the $\beta=0$ case from the supremum precisely to avoid having $0^\lambda=1$ for all limit $\lambda$.
