How does one refute this ultrafinitist argument? From Wikipedia: 

Edward Nelson criticized the classical conception of natural numbers because of the circularity of its definition. In classical mathematics the natural numbers are defined as $0$ and numbers obtained by the iterative applications of the successor function to $0$. But the concept of natural number is already assumed for the iteration. In other words, to obtain a number like $2\uparrow\uparrow\uparrow 6$ one needs to perform the successor function iteratively, in fact exactly $2\uparrow\uparrow\uparrow 6$ times to $0$.

Like most people, I am not an ultrafinitist. However, I do not quite know how to refute this argument. What are some criticisms of it? 
 A: I am agnostic about most questions of the philosophy of mathematics, but in this case, I think Nelson's argument is incoherent: a primitive shepherd who doesn't have any abstract concept of number can count sheep being herded into a fold by making marks on a stick and count the sheep out again by crossing the marks off (and this is how our human notions of cardinal numbers probably developed). There is no circularity involved.
To talk meaningfully about notations like $2 \uparrow\uparrow\uparrow 6$, you must have admitted a notion of definition by recursion that (as an ultrafinitist) Nelson can't accept. It seems to be me to be incoherent for an ultrafinitist to say anything more than "I can't accept that a primitive shepherd could ever have enough sticks".
A: An arithmetic theory (like Peano's) doesn't define what we understand by natural numbers, it is just an axiomatic theory over a formal deductive system, the circularity only appears when we try to associate each natural number to a specific term of the formal language the theory is based on, because to comunicate that association we should say something like "the term associated to the number TWO is the result of applying TWO times the successor function to the closed term denoted by the zero symbol". Here is where it is necessary to distinguish clearly which propositions belong to the theory and which ones belong to the meta-theory.
When you say "refute" you don't mean a formal refutation of Nelson's argument, since it isn't a formal argument, so there is not clear and objective way to determine if his argument is valid (we are not talking about logical validity)
A: This argument is wrong if it is intended to imply that the classical construction cannot be saved by arguing more carefully.
It is possible to have a reasonable theory of arithmetic without the axiom of infinity. It's just that the class of natural numbers won't necessarily form a set.
If you have the axiom of infinity, you can define natural numbers as follows. Say a set $A$ is inductive if: (i) $\varnothing \in A$; and (ii) whenever $x \in A$ we have $x^{+} \in A$, where $x^{+}$ is defined as $x \cup \{x\}$. The most common version of the axiom of infinity is "there exists an inductive set." Then you can define a set $n$ to be a natural number if it belongs to every inductive set. The axiom of infinity then implies that the class of all natural numbers indeed forms a set. We write $0 = \varnothing$, $1 = 0^{+} = \{0\}$, $2 = 1^{+} = \{0,1\}$, $3 = 2^{+} = \{0,1,2\}$,... (If you just look at this last line, then things do look circular—but here I am only naming the numbers, not defining the concept of number.) 
If the axiom of infinity is false, then according to the above definition, everything is a natural number. So an alternative approach is needed if you are agnostic about the axiom of infinity. A possible definition would then be as follows.
A set $n$ is called a natural number if 


*

*either $n = \varnothing$ or $\varnothing \in n$, and

*assume $A \subseteq n^{+}$, $\varnothing \in A$, and whenever $x \in A - \{n\}$, we have $x^{+} \in A$; then $A = n^{+}$.


Note that this definition is not at all circular as it does not involve the concept of natural number in any way. Moreover, the order relation $<$ and the operations $+$ and $\times$ can be defined and their basic properties proved using only set-theoretic axioms.
Simply put, natural numbers can be defined and arithmetic developed even in the absence of the axiom of infinity, provided certain changes are made to the way natural numbers are defined.
A: Nelson's critique is groundless. Peano's axioms are just a list of the essential properties of the set of natural numbers, essential in the sense that all other known properties can, in theory, be derived from these few properties. There is nothing "circular" about them. So successful have they been in this regard that these essential properties have come to define the set of natural numbers $\mathbb{N}$.
The natural numbers and many of their properties have been known for millennia across diverse cultures. Can any reasonable person deny that:

(1) $0$ is a natural number.
(2) Every natural number has a unique successor.
(3) Different natural numbers cannot have the same successor.
(4) No natural number has a successor that is $0$.

Now, some might quibble about the 5th axiom, the principle of mathematical induction:

(5) For all subsets $P \subset\mathbb{N}$, if (a) $0\in P$, and (b) for all $x\in P$, we also have the successor of $x$ being in $P$, then $P=\mathbb{N}$.

With only a bit more reflection, this, too, passes for a reasonable property of the set of natural numbers. There is nothing weird or "unnatural" about it. In fact, induction can be shown to hold on some subset of any non-empty set $X$.
Proof sketch: Let $X$ be an arbitrary, non-empty set. There exists at least one function $f: X \to X$, namely the identity function. Given any function $f:X\to X$ and $x_0\in X$, we can construct a subset $N\subset X$ such that $N=\{x_0, f(x_0) f(f(x_0)), \cdots \}$ and show that induction holds on $N$, namely that:

For all subsets $P \subset N$, if (a) $0\in P$, and (b) for all $x\in P$, we also have the successor of $x$ being in $P$, then $P=N$.

See my formal proof at http://www.dcproof.com/InductionMinRequirementsV2.htm
A: Two counterarguments I could think of are:


*

*The same argument could go for $1$, $2$ and $3$ , and I think that most ultrafinitists still think that numbers like $1$, $2$ and $3$ exist. 

*On the other hand the number $n$ is $n-1$ times the successor of $1$, so if $n-1$ and $1$ exist, $n$ should also exist using this argument. 

