Breaking up a countable sum Let $W$ be a countable set and $p:W\to[0,1]$ be any function satisfying
$$
\sum_{w\in W}p(w)=1.
$$
Now, let $A_1,A_2,\ldots$ be a countable collection of disjoint subsets of $W$. With $A=\bigcup_n A_n$, it is intuitive that
$$
\sum_{w\in A}p(w)=\sum_n \sum_{w\in A_n}p(w). \tag{*}
$$
Do I need to justify this seemingly obvious implication? If so, could you please show how? What are some circumstances (if they exist) that would make (*) fail?

Context: this comes from a theorem stated without proof in Rosenthal's A First Look at Rigorous Probability Theory.
 A: The equation (*) is true in a much more general setting (your index set could be uncountable as long as $p(w) \geq 0$ for all $w$). 
A scenario where ( * ) doesn't hold: For $w=1,2,...$, let $p(w)=(-1)^w$ and $A_n = \{2n-1,2n\}$. The left side of (*) is undefined while the right hand side is 0. Further a well-known theorem about conditionally convergent series, say $\sum_n a_n$, is that for any number $\alpha \in \mathbb{R}$, there is a rearrangement of $a_n$ such that $\sum_n a_{\sigma(n)} = \alpha$ [take $A_n = \{\sigma(n)\}$ here].
You can prove (*) using a standard liminf and limsup argument. Are you familiar with such techniques?
EDIT: First, Riemann Series Theorem says that for any conditionally convergent series and any $\alpha \in \mathbb{R}$ (actually in the extended reals), there is a rearrangement of the series that converges to $\alpha$. 
Now, let's show (*). First, since $W$ is countable, we may as well assume that $W = \mathbb{N}$. Note that
\begin{equation*}
Q:=\sum_n \sum_{w \in A_n} p(w) = \lim_{k \to \infty} \sum_{n=1}^k \lim_{\ell_n \to \infty} \sum_{w \in A_n, w \leq \ell_n} p(w).
\end{equation*}
Let $P(A) := \sum_{w \in A} p(w)$. Our goal is to show that $P(A)=Q$. 
Let $\epsilon>0$. Then there is some $N>0$ such that for any $k\geq N$
\begin{equation*}
P(A) - \epsilon \leq \sum_{j=1}^k p(w) \leq P(A)+\epsilon,
\end{equation*}
and in particular, $\sum_{j=1}^N p(w) \geq P(A)-\epsilon$. Note that for any $k$ and $\ell_1, \ldots, \ell_k$, we have that
\begin{equation*}
Q \geq \sum_{n=1}^k \sum_{w \in A_n, w \leq \ell_n} p(w),
\end{equation*}
since $p(w) \geq 0$. There is necessarily some $k (=k(N))$ so that 
\begin{equation*}
\{1, \ldots, N\} \subset \cup_{n=1}^k \left( A \cap \{1, \ldots, \ell_n:=N\right\}).
\end{equation*}
Therefore
\begin{equation*}
Q \geq \sum_{n=1}^k \sum_{w \in A_n, w \leq \ell_n} p(w) \geq \sum_{w=1}^N p(w) \geq P(A) - \epsilon.
\end{equation*}
Now we prove that $Q \leq P(A) + \epsilon$. It suffices to show that 
\begin{equation*}
\sum_{n=1}^k \lim_{\ell_n \to \infty} \sum_{w \in A_n, w \leq \ell_n} p(w) \leq P(A)+\epsilon
\end{equation*}
for all $k$ (use limsup here). Since $p(w) \geq 0$, it suffices to show that
\begin{equation*}
\lim_{m \to \infty} \sum_{n=1}^k \sum_{w \in A_n, w \leq m} p(w) \leq P(A)+\epsilon.
\end{equation*}
(Why? Pick $m=\max \ell_n$)
Finally, it is clear that
\begin{equation*}
\sum_{n=1}^k \sum_{w \in A_n, w \leq m}p(w) \leq \sum_{w=1}^m p(w) \leq P(A).
\end{equation*}
