Consider a measurable space $(\mathcal{X},\Sigma)$ with two probability measures P and Q on it such that $P<<Q$ (P is absolutely continuous with respect to Q). Under what conditions can the likelihood ratio $\frac{P(B)}{Q(B)}$ be replaced by the Radon-Nikodym derivate $\frac{dP}{dQ}$?

I'm confused because as these measures are $\sigma$-finite, they are non-atomic. But, then what does it mean to define the derivate as a density on the underlying space?

The overall context of this problem is in defining the Bayesian rule measure-theoretically, i.e. if the outcome space is discrete we know that the posterior distribution can be obtained by multiplying the likelihood with the prior over some parameter space $\Theta$: $p(\theta|x)=\frac{p(x|\theta)}{p(x)}\pi(\theta).$ Now, when the outcome space is continuous, under the measure-theoretic formalism the likelihood is not defined as $p(x)=0$. I'm wondering if it makes sense to replace $\frac{p(x|\theta)}{p(x)}$ by the corresponding Radon-Nikodym derivative.

  • 1
    $\begingroup$ Suppose that $\Sigma$ is generated by some finite disjoint collection $\cal F$ (i.e, any measurable set can be written as a finite disjoint union of members of $\cal F$). Then for each $B \in \cal F$ such that $Q(B)>0$, one has that $\frac{dP}{dQ}(\omega) = \frac{P(B)}{Q(B)}$ whenever $\omega \in B$. You can check this directly using the definition of conditional expectation. $\endgroup$
    – shalop
    May 25, 2018 at 5:50
  • $\begingroup$ But, I'm a bit confused. Since both measures are non-atomic, they assign zero probability to singletons. Then, does the definition of the derivative make sense as a function of x? $\endgroup$ May 25, 2018 at 7:28
  • $\begingroup$ Why do you think that $\sigma$-finite measures are non-atomic? That's obviously not true, and the example I gave is clearly a purely atomic measure. $\endgroup$
    – shalop
    May 25, 2018 at 7:47
  • $\begingroup$ That's right. A $\sigma$-finite measure needn't be non-atomic. It only assigns zero probability to singletons which shouldn't have anything to do with non-atomicity of the measure! Thanks for the clarification. $\endgroup$ May 25, 2018 at 8:28
  • $\begingroup$ No, that is still wrong. Sigma finite measures need not assign zero probability to singletons. Consider counting measure on $\Bbb Z$. $\endgroup$
    – shalop
    May 25, 2018 at 10:17


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.