Why are $p$-adic numbers and $p$-adic integers only defined for $p$ prime? It makes perfect sense to speak of a base $10$ digit expansion. Why does it not make sense to speak of $10$-adic numbers or $10$-adic integers?
 A: It does, but the result is not as nice. And we have $\mathbb{Z}_{10} = \mathbb{Z}_2 \times \mathbb{Z}_5$, so we can understand 10-adic numbers in by breaking them apart into a $2$-adic and a $5$-adic piece, each of which does behave nicely.
A: As has been said in other answers the 10-adic integers have zero-divisors. To sketch a proof of this, let me write $[m]_n$ for the equivalence class of $m$ modulo $n$, so $\Bbb{Z}/n\Bbb{Z} = \{[0]_n, [1]_n, \ldots[n-1]_n\}$.
Then $[m]_{2^i5^i} \mapsto ([m]_{2^i}, [m]_{5^i})$ is well-defined and defines an isomorphism $\phi_i$ say between the ring $\Bbb{Z}/10^i\Bbb{Z}$ and the product ring $(\Bbb{Z}/2^i\Bbb{Z}) \times (\Bbb{Z}/5^i\Bbb{Z})$. There is a unique pair of classes $x_i = [m_i]_{10^i}$ and  $y_i = [n_i]_{10^i}$, such that $\phi_i(x_i) = (1, 0)$ and $\phi_i(y_i) = (0, 1)$. By uniqueness $x_i = [m_{i+1}]_{10^i}$ and $y_i = [n_{i+1}]_{10^i}$. So you can form 10-adic integers $M$ and $N$ say such that for all $i$
$$M \equiv m_i \mod 10^i\\
N \equiv n_i \mod 10^i$$
and then as $m_in_i \equiv 0 \mod 10^i$ for all $i$, you will have $MN = 0$ in the ring of 10-adic integers.
A: You can define the $n$-adic integers, even if $n$ is not prime. For example, every $10$-adic integer has a $10$-adic decimal representation, and you can add them and carry as usual.
One thing that I like about this system is an alternate way of writing negative numbers in base 10. Instead of having to put a negative sign in front, we have $-1 = \ldots999$, $-2 = \ldots998$, and so on. Here is an example of computing $37 - 50 = -23$  in the $10$-adic numbers:
\begin{align*}
\ldots0000037& \\
+\quad \ldots 9999950 &\\
\hline 
= \quad\ldots 9999987&\\
\end{align*}
And here is $37 \cdot -50 = -1850$:
\begin{align*}
\ldots0000037& \\
\cdot\quad \ldots 9999950 &\\
\hline 
= \quad \ldots 00000000 &\\
+ \quad \ldots 0000185\phantom{0} &\\
+ \quad \ldots 000333\phantom{00} &\\
+ \quad \ldots 00333\phantom{000} &\\
+ \quad \ldots 0333\phantom{0000} &\\
+ \quad \ldots 333\phantom{00000} &\\
+ \quad \phantom{00} \cdots \phantom{000000} &\\
\hline
= \quad\ldots 99998150&\\
\end{align*}
Edit
Most of the other answers suggest that the reason we don't like the $10$-adic numbers, the reason they don't behave nicely is that they have zero divisors. But we are perfectly fine dealing with rings with zero divisors such as $\mathbb{Z} / 10\mathbb{Z}$. So let me elaborate on this.
A key part of any good introduction to $p$-adic numbers is a discussion of their topological properties, which come from the $p$-adic absolute value.
For example, this is how we may obtain the $p$-adic numbers as a completion of the rational numbers $\mathbb{Q}$ with a different distance metric than the one that gives $\mathbb{R}$.
The problem with zero divisors in this context is that a ring absolute value cannot even be defined when there are zero divisors. Why?
Because we really want it to be the case that $|xy| = |x| \cdot |y|$, so if $x$ and $y$ are zero divisors, then since $|0| = 0$, $|x| = 0$ or $|y| = 0$.
If we think of the $10$-adic integers as the ring $\mathbb{Z}_2 \times \mathbb{Z}_{5}$, this essentially forces us to either give the $\mathbb{Z}_2$ part of every $10$-adic integer zero norm, or to give the $\mathbb{Z}_5$ part of every integer zero norm. So the $10$-adic numbers do not have a natural norm. This is probably a big reason that we don't usually study them.
A: You can speak about $10$-adic numbers just fine, but they don't behave as nicely as $p$-adic numbers. For example, the $10$-adic integers have zero divisors, so no matter how you complete or massage them afterwards, you won't be able to embed them in a field.
