Systems the Peano axioms can be derived in I know that the Peano axioms can be got from ZF set theory, and lambda calculus. What other systems can they be derived (I'm not sure if that is quite the right word here) in, and which the simplest system?
 A: It seems you're interested in formal system capable of interpreting(encode) arithmetics and prove Peano Axioms. That is formal system in which you can define (via first order formulas) what are a natural numbers and arithmetic operations, and then you can prove PA axioms inside this system.
This can clearly be done in any sufficiently strong system, for instance every set theory rich enough to be a foundational theory: this include set theories like ZFC, but also NBG and TG, but also formal systems like lambda calculus and type theories.
Of course the simpler system in which you can encode Peano Arithmetic is Peano itself.
But if you are looking for a non trivial example you can consider ZF without the axiom of infinity. Such theory you can encode natural numbers as finite ordinals (as done in ZFC) and arithmetic operations as some predicates of this theory. Then every axiom of (first order) Peano Arithmetic can be proved by the axioms of this set theory. This is intuitively the theory of finite set and is the weakest system that can encode (first order) Peano Arithmetic: that's because it can be proved that ZF without the axiom of infinity can be encoded in Peano Arithmetic and so it's equivalent to it. 
A: Let $X$ be any Dedekind-infinite set, i.e. there exists a function $f:X\to X$ such that $f$ is injective, but not surjective. $f$ not being surjective, there exists $x_0\in X$ such that for all $y\in X$, we have $f(y)\neq x_0.$  
It can be shown using ordinary logic and set theory that there exists $N\subset X$ on which the Peano axioms are satisfied with $f$ being the successor function and $x_0$ being the "first" number in $N$. 
In other words, there exits at least one "copy" of the natural numbers in every Dedekind-infinite set.
A: Dan Christensen's answer ends with a fundamental truth:

...there exits at least one "copy" of the natural numbers in every
  Dedekind-infinite set

Although this answer uncovers the same fundamental truth, it is interesting that we can initiate the project from a purely algebraic perspective.
Let a set $M$ be endowed with a binary operation $+$ satisfying the following properties:
$\quad$ P-1: $\exists\, 1 \in M\, \text{ s.t. } [K \subset M \text{ and } 1 \in K  \text{ and } K \text{ is closed under } +] \implies K = M$
$\quad$ P-2: $\forall \; m,n \in M \text{, } \;1 + m = 1 + n \implies m = n$
$\quad$ P-3: $\forall \; m \in M \text{, } \;1 \ne  1 +  m$
Note that we assume nothing about the binary operation, but using the $+$ notation suits our purposes here.
We will show that $(M,+)$ is, up to isomorphism, the set of natural numbers $\mathbb N$. It is intuitively more than plausible if you try creating 'words' in a one letter alphabet (with no restrictive relations).
We define the 'increment by 1' function $\sigma: M \to M$ by
$\tag 1 m \mapsto m + 1$
By $\text{P-3}$, $1$ is not in the range of $\sigma$. Let $\mathscr I$ be the set of subsets $I$ of $M$ satisfying 
$\tag 2 1 \in I$
$\tag 3 \sigma(I) \subset I$
Now $M$ belongs to $\mathscr I$ and so does the set $M^{♯}$ defined by
$\tag 4 M^{♯} = \bigcap_{I \in \mathscr I} I$
So by construction,  $M^{♯}$ is the minimal set that remains invariant under $\sigma$. Expressed another way:
If $K \subset M^{♯}$, $1 \in K$, and whenever $k \in K$, $\sigma(k) \in K$, then $K$ must necessarily be equal to $M^{♯}$.
The following result is easily demonstrated.
Theorem 1: The function $\sigma: M^{♯} \to M^{♯}$ satisfies the Dedekind–Peano axioms.
Using the Peano construction, we can  recursively build up an addition operation 
$\tag 5 m + \sigma(n) = \sigma(m + n)$
on $M^{♯}$ that 'turns it into' $(\mathbb N,+)$.
Now the following might be viewed as a 'hair-splitting' algebraic point, but some may find it of interest.
Theorem 2: The set $M^{♯}$ is closed under the addition operation from  $(M,+)$. Moreover, the two binary operations agree.
Proof (sketch)
The identity mapping $\iota: (M^{♯},+_\sigma) \to (M^{♯},+)$ satisfies
$\quad \iota: 1 \mapsto 1$
$\quad (\iota \circ \sigma)(m) = (\sigma \circ \iota)(m)$
To show $\iota$ is indeed a morphism, observe, that, using simultaneous induction on both $u$ and $v$,
$\quad \iota ((u+_{\sigma}1) +_{\sigma} v) = (\iota \circ \sigma) (u +_{\sigma} v) =  (\sigma \circ \iota)(u+_{\sigma}v)=$
$\quad \quad \sigma(\iota(u) + \iota(v))= (\iota(u) + \iota(v)) + \iota(1) = (u+v) + 1 $
So the image of $\iota$ is closed under $+$, and since $1 \in (\iota(M^{♯}), +)$, by $\text{P-1}$ $M^{♯} = M$.
$\quad \blacksquare$
