Natural numbers in set theory is $\{0,1,2,...\}?$ The set of natural numbers $\mathbb{N}$ in set theory is defined with the axiom of infinity as the smallest inductive set and then it is usually proven that $\mathbb{N}$ satisfies the Peano axioms and well-ordering. However, I have yet to see a treatment where it is proven that $\mathbb{N}$ is equal to $\{0,1,2,...\}$, i.e. $\mathbb{N}$ contains the successors of 0 and nothing else.
I guess to answer my question, I would need to construct a model of set theory in which $\mathbb{N}$ contains $0$, $1$, $2$, etc. plus some extra junk and show that this model indeed works. Maybe someone has done this already? Or maybe it can be proven directly?
Edit: There seems to be some confusion in the comments and answers below. After some thought, I believe that what I want to show is that $\mathbb{N}$ does not contain an element apart from 0 that is not the successor of anything else in $\mathbb{N}$. I think I can prove this by contradiction with the axiom of specification to get an inductive set smaller than $\mathbb{N}$.
 A: Instead of working "backwards", work "forward". In that case, the need for the axiom of infinity is also mitigated.
Begin by defining ordinals, or cardinals whichever you like. Ordinals work better since they have an inherent successor operation. Now define the notion of "finiteness", it is true that many times this notion makes an appeal to the natural numbers but there are purely set theoretical definitions of finiteness. For example Dedekind-finiteness, Tarski-finiteness, and others.
Now define $\omega$ as the collection of all finite ordinals. This is exactly what you get when you begin with $\{0\}$ and close it under the successor operation. Next show, if you like, that this set is inductive and that it is in fact minimal.
In the course I'm TA'ing right now, the professor (who is a distinguished set theorist) is taking this sort of approach. The natural numbers are some atomic object at first, and after defining the ordinals we take the finite ordinals, $\omega$ as the set theoretic natural numbers. This circumvents the need to talk about inductive sets at all.
A: Question (quote) "I want to show that $\mathbb{N}$ does not contain an element apart from 0 that is not the successor of anything else in $\mathbb{N}$."
Short Answer This would contradict the minimality of $\mathbb{N}$. You mentioned a correct proof sketch by contradiction.
Long Answer You asked if it can be proved directly; we will show that every element of $\mathbb{N}$ is either $0$ or an iterated-successor of $0$ (i.e. $0$ is an element). This answers your overall question of why the alternative notation $\{0,1,2,\ldots\}$ is justified.
Remark "$\ldots$" is nothing but a symbol---it is meaningless and only used in notations. It's much like "$\infty$" in that regard. Another note is that the ZFC axioms do not restrict us on how to denote our sets. As long as we're clear about which unique set we refer to, we can use any notation for any set; i.e. $\omega$ and $\aleph_0$ are notations for the exact same set even though they're used in different contexts.
Recall

$\mathbb{N}$ is the minimal inductive set; i.e. it's an inductive set such that for each inductive set $B$, if $B\subseteq \mathbb{N}$, then $B=\mathbb{N}$.

Brief review

$A$ is an inductive set if $\varnothing\in A$ and for each $X\in A$, $X\cup\{X\}\in A$.
A natural number is an element of $\mathbb{N}$.

We will use "$+1$" to denote successor; i.e. $X+1=X\cup\{X\}$.
From this we can derive the following (induction)

For each formula $\phi(x)$, if $\phi(0)$ holds and for each natural number $k$, if $\phi(k)$ holds, then $\phi(k+1)$, then for each natural number $n$, $\phi(n)$ holds.

At this point most would be satisfied that the notation $\{0,1,2,\ldots\}$ is justified, but we want to be pedantic and use the above theorem

Define the formula $\phi(x)$ by$$\phi(x)\overset{\mathrm{def}}{=} (x\ne 0)\Rightarrow (0\in x)$$

This is something that can be proved by induction for each natural number.
(sorry for not posting proofs, but you already have one)
A: The Peano Axioms contain the Axiom of Induction, which says, basically, that the natural numbers contain only $0$ and its successors. At least, what I understand you to mean when you say "the successors of $0$", because, and Trevor pointed out, $0$ has only one successor. One formal way of stating this axiom is that if there is a set $S$ where $0\in S$, and for any natural number $n$, if $n$ is in $S$, then the successor of $n$ is in S, then the set $S$ contains every natural number. In other words, the statements "$0$ is a natural number" and "if $n$ is a natural number, then the successor of $n$ is a natural number," when combined with the other axioms about what the successor is and why $0$ is special, define all of the natural numbers.
So, if you show that $\Bbb{N}$ satisfies the Peano Axioms, then "$\Bbb{N}$ contains the successors of $0$ and nothing else." The Axiom of Induction is in the Peano Axioms to exlude that "extra junk".
See this question.
A: 
I guess to answer my question, I would need to construct a model of set theory in which N contains 0, 1, 2, etc. plus some extra junk and show that this model indeed works. Maybe someone has done this already? Or maybe it can be proven directly?

You can use the compactness theorem for first-order logic to construct such a model for ZFC, under the assumption that ZFC is consistent (i.e. has any models at all). A fun example is constructing a model whose $\mathbb{N}$ contains an infinite descending chain.
Add constants $N, c_0, c_1, c_2, \dots$ to the language of ZFC. Extend ZFC to  premise $\Gamma$ by adding the following family of sentences:
"$N$ is the set of natural numbers", i.e. something like $N = \bigcap\{ x\mid x\text{ is an inductive set}\}$
$c_0 \in N$
$c_1 \in N$
$c_2 \in N$
$\vdots$
$c_0 > c_1$
$c_1 > c_2$
$c_2 > c_3$
$\vdots$
Consistency of ZFC implies that there is some model $M$ which satisfies ZFC. If $\Gamma_0 \subseteq \Gamma$ is finite, then there is a maximum $n$ such that the sentence $c_{n-1} > c_n$ is in $\Gamma_0$, so we can extend $M$ by (denoting $M$'s naturals by $0_M$, $1_M$, etc.) assigning:
$0_M$ to $c_n$,
$1_M$ to $c_{n-1}$, 
$\vdots$
$(n-1)_M$ to $c_1$
$n_M$ to $c_0$
By design this assignment to constants $c_0, \dots, c_n$ satisfies all the sentences of form $c_i > c_{i+1}$ in $\Gamma_0$. We can satisfy all sentences of form $c_i \in N$ by, say, setting $c_i$ to $0_M$ for all $i > n$. As we've made no other changes to $M$, the finite subset of $ZFC$ contained in $\Gamma_0$ continues to be satisfied by this model, so we have a model which satisfies $\Gamma_0$.
Since we can do this for any finite $\Gamma_0 \subseteq \Gamma$, it follows from the compactness theorem that $\Gamma$ itself must have a model $M'$, within which there is now an infinite descending chain of naturals $c_0 > c_1 > c_2 > \dots$
As $\Gamma$ is a superset of $ZFC$, it follows that $M'$ is a model of $ZFC$ itself.
Also of note, facts like $\forall x\forall y(x\in N \wedge y \in N \wedge x < y \rightarrow x \in y)$ continue to hold in $M'$, so $c_0 \ni c_1 \ni c_2 \ni \dots$ is actually an infinite descending chain of set memberships, in apparent violation of the axiom of regularity. The rub is that $M'$ doesn't "know" about this sequence -- you won't find a function $f$ within $M'$ itself that has $f(i) = c_i$ for all $i \in \mathbb{N}$, even though there is clearly such a function in the observer language.
