The question needs precision on what you mean by $\mathbb{Z}_{51}$. Since you are studying Atiyah-Macdonald I assume what you meant to ask about is the completion of $\mathbb{Z}$ with respect to the ideal $51\cdot\mathbb{Z} \subset \mathbb{Z}$. If you mean with respect to some "metric" then you have other isues as quid pointed out.
But, using the definition of completion given in AM, Chapter 10, you get nothing more serious by considering composite numbers. Indeed, if $m$ and $n$ are coprime then $\mathbb{Z}/(mn)^r\mathbb{Z} \simeq \mathbb{Z}/m^r\mathbb{Z} \oplus \mathbb{Z}/n^r\mathbb{Z}$ in a canonical way (i.e. compatible with changing the $r$) and so the completion of $\mathbb{Z}$ with respect to the ideal $mn\cdot \mathbb{Z}$, which maybe I will call $\mathbb{Z}_{mn}^{\wedge}$ for now, is naturally isomorphic to the product ring $\mathbb{Z}_m^{\wedge} \times \mathbb{Z}_n^{\wedge}$.
So, for example, the 51-adic completion of $\mathbb{Z}$ is just the product of the $3$-adic integers $\mathbb{Z}_3$ and the $17$-adic integers $\mathbb{Z}_{17}$.
I should say things like this show up all the time in real life, e.g. the profinite completion $\hat{\mathbb{Z}}$ of $\mathbb{Z}$ is a natural enough object but it is really just the product of the $p$-adic integers over all primes $p$.