Almost sure convergence of a sum of random variables Suppose $(X_i)_{i=1}^{\infty}$ is an i.i.d. sequence of rv's, where $X_i$ can take countably many values $\{x_1,x_2,\dots\}$ with probabilities $\{p_1,p_2\dots\}$, respectively. Let $p_{n,k}:= 1/n\sum_{i=1}^n \mathrm{1}\{X_i=x_k\}$, for $k\in\mathbb{N}$. By the SLLN, we have $p_{n,k} \rightarrow p_k,\forall k$, almost surely, as $n\rightarrow \infty$. 
Here is my question: How do I show that $\sum_{k=1}^{\infty}|p_{n,k}-p_k|\rightarrow 0$, almost surely, as $n\rightarrow \infty$?
I was thinking of using Borell-Cantelli, but could not make it work. Any help is much appreciated. many thanks!
 A: With probability $1$, you know that $p_{n,k} \to p_k$ as $n \to \infty$ for all $k$. Let us focus on this set (throw out the set of measure $0$ where we don't have convergence for all $k$. If you like, there are a countable number of such sets you need to throw out, so what you throw out has measure zero, thankfully). This problem no longer needs Borel Cantelli, it becomes regular analysis: 
Let me ask the following: suppose you know $p_{n,k} \to p_k$, how can you show that $\lim_{n\to\infty} \sum_k |p_{n,k} - p_k| = 0$, with the additional assumption that $\sum_{k} p_{n,k} = \sum_{k} p_k = 1$? This is probably what mike means when he says forget about the probabilities (restrict your attention to the event that $p_{n,k} \to p_k$ for all $k$, as this occurs with probability $1$).
A: Fresh slate now.
It suffices to show that for any $\epsilon > 0$, $\lim_{n\to\infty} \sum_k |p_{k,n} - p_k| \leq \epsilon$ with probability $1$.
For some $N$ to choose later, decompose with triangle inequality $\sum_k |p_{k,n}-p_k| \leq \sum_{k\leq N} |p_{k,n}-p_k| + \sum_{k>N} p_{k,n} + \sum_{k>N} p_k$.
As a random variable, note that as $n \to \infty$, $\sum_{k>N} p_{k,n}$ converges a.s. to $r_N := \sum_{k>N} p_k$ by law of large numbers. Thus, we have that taking the limit in $n$,
$\lim_n \sum_k |p_{k,n} - p_k| \leq 2r_N$ with probability $1$ (here we applied law of large numbers on the finitely many terms as well). All that is left is to choose $N$ so that $2r_N  < \epsilon$, which can be done because $r_N \to 0$ as $N \to \infty$.
