Where do $p$-adic numbers and $p$-Sylow theory both appear? Both $p$-adic numbers and $p$-Sylow theory are by design "arithmetic" ways of "localizing," so it stands to reason they might be in cahoots in certain contexts. Are they? 
 A: Here's one reasonably simple link:  In terms of Sylow theory, a nice home for generalization is the land of profinite groups, where you have directly analogous Sylow theorems (with directly analogous proofs).  Among profinite groups there is the canonical $\widehat{\mathbb{Z}}$, the profinite completion of $\mathbb{Z}$ and free profinite group of rank 1.  The decomposition
$$
\widehat{\mathbb{Z}}\cong \prod_p \mathbb{Z}_p
$$
shows that the $p$-adic integers are simply the $p$-Sylow subgroup of this very canonical group.
A: In the context of abelian torsion groups, Sylow subgroups correspond to the summands occurring in the torsion decomposition as per structure theorem. In particular, as additive groups,
$$\frac{\bf Q}{\bf Z}\cong \bigoplus_p \frac{{\bf Q}_p}{{\bf Z}_p}.$$
The Prufer $p$-groups ${\bf Z}(p^\infty)\cong{\bf Q}_p/{\bf Z}_p$ and the $p$-adic integers ${\bf Z}_p$ are Pontryagin duals of each other. This decomposition is the number field analogue of partial fraction decomposition present in function fields, which I elaborate over at this answer, illustrating this is a global phenomenon.

The connection between nilpotence and Sylow subgroups allows zeta function techniques to study  finitely-generated torsion-free nilpotent (i.e. $\frak T$-)groups, in particular lattice-of-subgroup structure and asymptotic analysis of subgroup growth, via local Euler products. The $p$-adic numbers come into play when local zeta integrals are used. See image, source being the first reference below.


*

*Zeta Functions of Groups and Rings - du Sautoy & Woodward

*Lectures on Profinite Topics in Group Theory - Klopsch, Nikolov & Voll



Here is a connection I have not seen mentioned anywhere. Let $W_{p^n}$ be the wreath power $$W_{p^n}=\underbrace{C_p\wr C_p\wr\cdots\wr C_p}_n.$$
The Sylow $p$-subgroups of $S_n$, where $n=a_kp^k+\cdots+a_1p+a_0$ is $n$'s $p$-adic expansion, are
$$a_kW_{p^k}\oplus\cdots\oplus a_1 W_{p}\oplus a_0 W_1,$$
up to isomorphism, where $W_1$ is the trivial group and $$mG:=\underbrace{G\oplus G\oplus\cdots\oplus G}_m.$$ This can be proven by computing the order of the above subgroup in $S_n$. Note that $W_{p^n}$ is also the automorphism group of the $p$-ary rooted tree of depth $n$. Incidentally, the topology on ${\bf Z}_p$ is that of an infinite-depth rooted $p$-ary tree (see Pictures of Ultrametric Spaces by Holly). 
As an aside, in Tree Representations of Galois Groups, Boston conjecturally notes that $p$-adic Galois representations into matrix groups convey very little information whereas $p$-adic Galois reps into tree automorphism groups should convey a lot, according to Hausdorff dimension.
Since isometries of ${\bf Z}_p$ correspond to automorphisms of the underlying tree topology, the isometry group is given by ${\sf Iso}({\bf Z}_p)\cong\varprojlim W_{p^n}$. The torsion subset is a subgroup isomorphic to $\varinjlim W_{p^n}$. The Sylow $p$-subgroup of ${\sf Sym}({\bf Z})\ge\varinjlim S_n$ is also seen to be $\varinjlim W_{p^n}$. Its topological closure (in terms of pointwise convergence of functions) is isomorphic to $\varprojlim W_{p^n}$. (See direct / inverse limits.) (At least I'm pretty sure about that last claim. Still trying to pin down my intuition into a proof.)
