If $\mu$ is a probability measure on $\mathbb R$, is $t\mapsto\mu\{t\}$ differentiable almost everywhere? In this question, the OP at one point says that since $t\mapsto P[X=t]$ is zero almost everywhere, it is almost everywhere differentiable. I pointed out that this need not be the case - $\mathbf1_{\mathbb Q}$ is the usual example of a function which is zero almost everywhere but continuous nowhere - but I realized the situation is a little more subtle in this case.
If $\mu$ is a probability measure on $\mathbb R$, then for every $\varepsilon>0$, $\mu\{t\}\ge\varepsilon$ for at most finitely many $t$. In particular, $\lim_{s\to t}\mu\{s\}=0$ for all $t\in\mathbb R$, so $t\mapsto\mu\{t\}$ is continuous at every point of continuity of $\mu$. (Recall that point of continuity refers to the function $t\mapsto\mu((-\infty,t])$, so this statement isn't as meaningless as it appears.) Since there can be at most countably many atoms, this shows that at the very least we do have a.e. continuity.
What about differentiability? This is a messier question. Obviously the function is not differentiable at any atom. If the derivative exists at $t_0$, then it must equal zero. We therefore have that differentiability at $t_0$ is equivalent to
$$\mu\{t_0+h\}=o(h)\qquad\text{as }h\to0.\qquad\qquad(\dagger)$$
Does $(\dagger)$ occur for almost every $t_0$? I don't know. I also don't know how one would prove it if they did know, either.  I had an idea for a proof of the affirmative statement, trying to prove that the set of $t$ where $\mu\{t\}=0$ but $(\dagger)$ fails is at most countable, but it didn't pan out. A counterexample would surely be messy and hard to visualize - it would need to have a countable set of atoms whose closure had nonempty interior at a minimum.
If the answer turns out to be negative, how badly behaved can the function be? Can the set $\{t:s\mapsto\mu\{s\}\text{ is differentiable at }t\}$ have zero measure?
 A: Let $f(t) = \mu \{ t \}$. 
Suppose $\mu \{ x \} = 0$ and $f'(x) \neq 0$ (by which we mean $f'(x)$ might not exist). Then there is $\varepsilon > 0$ such that arbitrarily close to $x$ there exists $y$ with $\mu \{ y \} \geqslant \varepsilon |y-x|$. 
There is at most countably many $y$ with $\mu \{ y \} > 0$ - if there is finitely many, of course $f$ is a.e. differentiable, otherwise arrange all such $y$ into a sequence $y_m$. Then $$\sum_{m=1}^{\infty} \mu \{ y_m \} = \mu( \{ y_m : m \geqslant 1 \} ) \leqslant 1.$$ 
Now, for $\varepsilon > 0$ let $I_m^{\varepsilon} = \{ x \in \mathbb{R} : \mu \{ y_m \} \geqslant \varepsilon |y_m-x| \}$ so $I_m^{\varepsilon}$ is an interval of radius $\frac{\mu \{ y_m \}}{\varepsilon}$ centered at $y_m$. By the remark in the beginning
$$\{ x \in \mathbb{R} : \mu \{ x \} = 0 \ \& \ f'(x) \neq 0 \} \subseteq \bigcup_{\varepsilon > 0} \bigcap_{n=1}^{\infty} \bigcup_{m=n}^{\infty} I_m^{\varepsilon}.$$
Now, for a fixed $\varepsilon > 0$ and $n \geqslant 1$ we have 
$$\lambda \left( \bigcup_{m=n}^{\infty} I_m^{\varepsilon} \right) \leqslant \sum_{m=n}^{\infty} \lambda( I_m^{\varepsilon} ) = \frac{2}{\varepsilon} \sum_{m=n}^{\infty} \mu \{ y_m \}.$$
As the series $\displaystyle \sum_{m=1}^{\infty} \mu \{ y_m \}$ converges, the above tends to $0$ as $n \to \infty$. Therefore
$$\lambda \left( \bigcap_{n=1}^{\infty} \bigcup_{m=n}^{\infty} I_m^{\varepsilon} \right) = 0$$
and since the union over $\varepsilon > 0$ can be made countable by taking it over a sequence $\varepsilon_k = \frac{1}{k}$, we obtain 
$$\lambda \big( \{ x \in \mathbb{R} : \mu \{ x \} = 0 \ \& \ f'(x) \neq 0 \} \big) = 0.$$
Finally, if $\mu \{ x \} \neq 0$, then $x = y_m$ for some $m$, so 
$$\lambda \big( \{ x \in \mathbb{R} : \mu \{ x \} \neq 0 \} \big) = 0.$$
Therefore
$$\lambda \big( \{ x \in \mathbb{R} : f'(x) \neq 0 \} \big) = 0,$$
i.e. $f'(x) = 0$ almost everywhere.
Example
Recall the construction of the Cantor set:
$$\mathcal{C} = \bigcap_{n=1}^{\infty} C_n$$ 
where each $C_n$ consists of $2^n$ intervals of total length $\left( \frac{2}{3} \right)^n$. Arrange them all in a sequence $I_m$ and let $y_m$ be the middle of $I_m$. Note that $y_m$ are pairwise distinct.
Now define a measure $\mu$ on $\mathbb{R}$ to be $\frac{1}{2}|I_m|$ on $y_m$ and $0$ on $\mathbb{R} \setminus \{ y_m : m \geqslant 1 \}$. Note that $\mu$ is a probability measure and each $I_m$ is exactly $I_m^1$ from the proof above. Since each $a \in \mathcal{C}$ belongs to intervals $I_m^1$ of arbitrarily small length, $f$ is not differentiable in any point $a \in \mathcal{C}$. 
The construction can be slightly modified to replace the Cantor set with a dense $G_{\delta}$ set.
Summary
For arbitrary probability measure $\mu$ on $\mathbb{R}$, the set of points where $t \mapsto \mu \{ t \}$ is not differentiable:


*

*must have measure zero;

*may contain the Cantor set;

*may contain a dense $G_{\delta}$ set. 
Also, as the OP has noted, wherever the derivative exists, it must be zero. 
