If $ X^2 -a$ has a zero in $\mathbb{Z}_{p^k}$ implies also zero in $\mathbb{Z}_{p^{k+1}}$? 
Let $p$ be a odd prime and $f = X^2 -a \in \mathbb{Z}[X]$. If $f$ has
  a zero in $\mathbb{Z}_{p^k}$ does $f$ have a zero in
  $\mathbb{Z}_{p^{k+1}}$?

I'm not exactly sure how to show this but I'm thinking Hensels lemma;
Assume theres a zero $\xi$ of $f$ in $\mathbb{Z}_{p^k}$ then $p^k$ divides $(\xi^2 -a)$
Then I would like to show that $p$ does not divide $f'(\xi) = 2\xi$, i.e there exists no $n\in \mathbb{Z}$ such that
$$pn = 2\xi$$
But here I'm stuck I dont get why there cannot be a factor $p$ in $\xi$.
Any help would be highly apprecaited.
 A: The problem is that if $a=0\in\Bbb Z/p^k\Bbb Z$, then of course $X^2-a$ has a solution in $\Bbb Z/p^k\Bbb Z$. Just avoid this situation, and everything goes smoothly.
So the strongest result is this: If $a$ is prime to $p$ and $X^2-a$ has a solution in $\Bbb Z/p\Bbb Z$, i.e. if $a$ is a nonzero square modulo $p$, then $a$ is a square in the $p$-adic integers $\Bbb Z_p$ (and now you see why I couldn’t use your notation for $\Bbb Z/p^k\Bbb Z$). A $p$-adic integer stands for a consistent sequence $(a_k)_k$ of elements of $\Bbb Z/p^k\Bbb Z$ such that for each $k\ge1$, $a_{k+1}\equiv a_k\pmod{p^k}$. You get this from Hensel’s Lemma, whether the strong form that I like, or the weak form that talks about roots and derivatives. The form I like says that if $f\in\Bbb Z_p[X]$. reducing to $\tilde f=\gamma\eta$ in $(\Bbb Z/p\Bbb Z)[X]$, where the factors $\gamma(X)$ and $\eta(X)$ are relatively prime, then you can lift them, preserving the degree of one of them, to $g(X)$ and $h(X)$, in such a way that $\tilde g=\gamma$ and $\tilde h=\eta$ and $gh=f$.
In particular, if $a\not\equiv0\pmod p$ and you have an integer $b$ with $b^2\equiv a\pmod p$, then the polynomial $X^2-\tilde a$ is equal to $(X-\tilde b)(X+\tilde b)$ as polynomials with coefficients in $\Bbb Z/p\Bbb Z$, and since $\tilde b\ne-\tilde b)$ (because of your assumption that $p\ne2$), then Hensel applies, and you can lift all the way up to a factorization $X^2-a=(X-B)(X+B)$, where $B$ is a $p$-adic integer, in particular you have good congruences modulo each power of $p$.
Now for the bad cases: if $a\in p\Bbb Z$, you must ask how divisible $a$ is by $p$. That is, write $a=p^ma_0$ where $a_0$ is prime to $p$. In case $m$ is odd, say $m=2n+1$, you’re out of luck, you can not find a root except when we’re in the problem case I mentioned at the top. For, you can solve $X^2-a$ modulo $p^m$, but not modulo $p^{m+1}$. I’ll let you calculate why. On the other hand, if $m$ is even, you look at $a_0$ and ask whether $X^2-a_0$ has a solution modulo $p$. If so, you’re good, and if not, you’re again out of luck. Again, I’ll let you fill in the details.
The story for $p=2$ is somewhat more complicated. If you need to know about this case, e-mail me.
A: If you know a bit about the field $\mathbb{Q}_p$ and its subring $\mathbb{Z}_p$, things become easier, at least formally.
Say $x_0^2 = a \mod p^k$. Well, let's assume in fact that the exponent of $p$ in $x^2-a$ is larger than the exponent of $p$ in $a$. 
$$x^2_0 = a + \delta'$$
with $e_p(\delta') > e_p(a)$.
Then 
$$\delta = \frac{\delta'}{a}$$
has 
$$e_p(\delta) >0$$
and we get
$$x_0^2 = a(1+\delta)$$
Fact: the element
$$x = x_0\cdot (1+\delta)^{-1/2}$$
satisfies
$$x^2 = a$$
where $(1+\delta)^{-1/2}$ is the sum of a series
$$(1+\delta)^{-1/2} = 1 +\frac{-1}{2} \delta + \frac{\frac{-1}{2}( \frac{-1}{2}-1)}{2} \delta^2 + \cdots  = \sum_{k\ge 0}\binom{(\frac{-1}{2})}{k}\delta^k$$
(since $p$ is odd and $e_p(\delta) >0$ this series is convergent- see also http://en.wikipedia.org/wiki/Binomial_theorem#Newton.27s_generalised_binomial_theorem)
$\bf{Added:}$
@Lubin: pointed out ( many thanks!) a neat way to rescale the relative error $\delta$ as rather $4 \delta$, since the series $(1+4 X)^{\frac{1}{2}}$ as a series in $X$ has all integer coefficients. Indeed:  recall the generating function for the Catalan numbers
$$\frac{1- \sqrt{1 - 4 X}}{2X} = \sum_{n\ge 0} C_n X^n$$ This rescaling imposes no extra coditions if $p$ is odd, and explains the case $p=2$. 
In general one can take
$$\frac{1 - ( 1- r^2 X)^{\frac{1}{r}}}{r X}= \sum C_{r,n} X^n$$ again a series with all integer coefficients if $r$ is an integer (generalized Catalan numbers). This would take care of $r$th roots. 
