Why $p$-adically interpolate? I'm studying $p$-adic analysis now and particularly $p$-adic interpolation; for example, constructions like $p$-adic $L$-functions (Kubota-Leopoldt style). I'm having some difficulty though, and I'd like to start with knowing what motivates the study. What do you really do with $p$-adic zeta functions and $L$-functions?
 A: This is not a question that has a short answer, because $p$-adic $L$-functions are one of the pivotal objects in modern number theory.
Let me try, though:
Riemann proved that the zeta function assumes rational values at negative odd integers.  The same sort of results holds for special values of Dirichlet $L$-functions.
Let us consider Dirichlet $L$-functions $\sum_{n = 1}^{\infty} \chi(n) n^{-k}$
where $\chi$ is a Dirichlet character modulo some $p^N$.  (If $\chi$ is trivial,
then this gives the "$p$-deprived" zeta function, i.e. the Riemann zeta function, but with Euler factor at $p$ removed.)
Now we can think of $n \mapsto \chi(n) n^{-k}$ as a character on $(\mathbb Z/p^N)^{\times}$ with values in $\mathbb Z[\zeta]/p^N$ (where $\zeta$ is
an appropriate root of unity chosen so that the values of $\chi$ land
in $\mathbb Z[\zeta]$).  
If $\chi,k$ and $\chi',k'$ are such that the two characters $n \mapsto \chi(n)n^{-k}$ and $n \mapsto \chi'(n) n^{\prime -k}$ are congruent mod $p^a$ (for some $a \leq N$), then looking at the Dirichlet series you might imagine that the corresponding $L$-values $L(\chi,k)$ and $L(\chi',k')$ 
should also be congruenct mod $p^a$ (just because the individual terms are congruent).
Of course reasoning this way with congruences isn't valid for infinite series,
and if $k$ and $k'$ are in the half-plane of convergence of the $L$-functions,
then the corresponding $L$-values are transcendental and congruence doesn't even make sense.
But ... at values of $k$ and $k'$ where the values are rational numbers, the congruence does hold!  Of course the above "reasoning" from a term-by-term congruence of the members of the sequence is bogus, both because it is not valid for an infinite series, and the series doesn't converge at these negative values of $k$ anyway.  But one can still prove the congruence.  (Kummer did the case $a = 1$, i.e. just working mod $p$.  Kubota and Leopoldt did the general case.)
Kummer's motivation was (among other things) that the case when $k = 0$ and $\chi$ is non-trivial has to do with the class numbers of $p$-adic cyclotomic fields (by the class number formula) and he wanted to know when these were divisible by $p$ (i.e. when primes were irregular).  Using his congruence, 
he could instead test this by working with the trivial character but with a non-zero (negative) value of $k$.  But in this case we are just looking at special values of the $\zeta$-function, which are Bernoulli numbers.  This is how Kummer proved his criterion for a prime to be regular in terms of Bernoulli numbers. 
So the theory of $p$-adic $L$-functions has its roots in Kummer's study of cyclotomic fields.  It continues to be a central topic in the study of cyclotomic fields, and, generally speaking, $p$-adic $L$-functions form the basis for almost all modern investigations into formulas relating $L$-values and arithmetic (such as the BSD conjecture, the main conjecture of Iwasawa theory in various contexts, the  Bloch--Kato conj., etc.) 
