Consider a measurable space $(\mathcal{X},\Sigma)$ with two probability measures P and Q on it such that $P<<Q$ (P is absolutely continuous with respect to Q). Under what conditions can the likelihood ratio $\frac{P(B)}{Q(B)}$ be replaced by the Radon-Nikodym derivate $\frac{dP}{dQ}$?
I'm confused because as these measures are $\sigma$-finite, they are non-atomic. But, then what does it mean to define the derivate as a density on the underlying space?
The overall context of this problem is in defining the Bayesian rule measure-theoretically, i.e. if the outcome space is discrete we know that the posterior distribution can be obtained by multiplying the likelihood with the prior over some parameter space $\Theta$: $p(\theta|x)=\frac{p(x|\theta)}{p(x)}\pi(\theta).$ Now, when the outcome space is continuous, under the measure-theoretic formalism the likelihood is not defined as $p(x)=0$. I'm wondering if it makes sense to replace $\frac{p(x|\theta)}{p(x)}$ by the corresponding Radon-Nikodym derivative.
• Suppose that $\Sigma$ is generated by some finite disjoint collection $\cal F$ (i.e, any measurable set can be written as a finite disjoint union of members of $\cal F$). Then for each $B \in \cal F$ such that $Q(B)>0$, one has that $\frac{dP}{dQ}(\omega) = \frac{P(B)}{Q(B)}$ whenever $\omega \in B$. You can check this directly using the definition of conditional expectation. May 25, 2018 at 5:50
• Why do you think that $\sigma$-finite measures are non-atomic? That's obviously not true, and the example I gave is clearly a purely atomic measure. May 25, 2018 at 7:47
• That's right. A $\sigma$-finite measure needn't be non-atomic. It only assigns zero probability to singletons which shouldn't have anything to do with non-atomicity of the measure! Thanks for the clarification. May 25, 2018 at 8:28
• No, that is still wrong. Sigma finite measures need not assign zero probability to singletons. Consider counting measure on $\Bbb Z$. May 25, 2018 at 10:17