10
$\begingroup$

I've been self-studying measure theory from Richard Ash's textbook, and am close to completing Chapter 2 (so, for instance, I've covered the Radon-Nikodym Theorem).

What is an example of a measure which is useful for some application for which there is not a density with respect to either counting measure or Lebesgue measure?

I would be especially interested to see an example that is a probability measure.

I've seen it claimed (e.g. by David Pollard) that such examples are prevalent in practice (and that the intuition of statisticians can be unnecessarily restricted by considering probability distributions merely in terms of those with probability density functions or probability mass functions). But I can think of no such example at this point in my study.

$\endgroup$
3
  • $\begingroup$ By applications, do you mean real-world or within the realm of "pure" mathematics, or either? $\endgroup$
    – Alex Ortiz
    Apr 1, 2022 at 16:07
  • $\begingroup$ A Markov chain? It's a probability measure on a countable product of some space. It doesn't have a density. $\endgroup$
    – Mason
    Apr 1, 2022 at 16:24
  • $\begingroup$ @AlexOrtiz ideally applied mathematics rather than pure. sorry, i should have specified. $\endgroup$
    – ashman
    Apr 1, 2022 at 19:19

4 Answers 4

19
$\begingroup$

Let $\mu$ be a $\sigma$-finite Borel measure on $\mathbb{R}^d$. Then the Lebesgue decomposition theorem tells that $\mu$ is uniquely decomposed as the sum of three Borel measures:

$$ \mu = \mu_{\text{ac}} + \mu_{\text{sc}} + \mu_{\text{d}}, $$

where

  • $\mu_{\text{ac}}$ is absolutely continuous w.r.t. the Lebesgue measure $\operatorname{Leb}$ on $\mathbb{R}^d$, that is, $\mu_{\text{ac}}(\mathrm{d}x) = f(x) \, \mathrm{d}x$ for some non-negative Borel-measurable $f$ on $\mathbb{R}^d$.

  • $\mu_{\text{d}}$ is discrete, in the sense that it consists purely of point masses.

  • $\mu_{\text{sc}}$ is singular continuous, that is, $\mu_{\text{sc}} \perp \operatorname{Leb}$ but it has no point mass at all.

This theorem provides a good picture of how $\mu$ might fail to be either discrete or (absolutely) continuous. Typical scenarios include:

  1. $\mu$ is a mixture of both discrete and continuous part.

Example 1. Let $X$ be continuous with PDF $f_X$, and $Y = \min\{X, a\}$. Then $$ \mathbb{P}(Y \in \mathrm{d}x) = f_X(x)\mathbf{1}_{(-\infty, a)}(x) \, \mathrm{d}x + \mathbb{P}(X \geq a) \delta_a(\mathrm{d}x). $$

Example 2. If $X_0$ is discrete with PMF $p_{X_0}$, $X_1$ is continuous with PDF $f_{X_1}$, $Y $ has a Bernoulli distribution with parameter $p$, and if all of them are mutually independent, then $$ Z = X_Y \sim \begin{cases} X_1, & \text{with probability } p, \\ X_0, & \text{with probability } 1-p \end{cases}$$ is a mixture of discrete and continuous distribution: $$ \mathbb{P}(Z \in \mathrm{d}x) = p f_{X_1}(x) \, \mathrm{d}x + (1-p) \sum_a p_{X_0}(a)\delta_{a}(\mathrm{d}x)$$

  1. $\mu$ is singular continuous, being supported on a $k$-dimensional subset of $\mathbb{R}^d$ for some $0 < k < d$.

Example 1. The Cantor distribution is a singular continuous distribution supported on the Cantor set, which has Hausdorff dimension strictly between $0$ and $1$.

Example 2. The "uniform" distribution on the unit circle $\mathbb{S}^1 : x^2 + y^2 = 1$ is singular continuous in $\mathbb{R}^2$, since $\mathbb{S}^1$ is a $1$-dimensional subset of $\mathbb{R}^2$.

Example 3. A $d$-dimensional normal vector $X \sim \mathcal{N}(\mu, \Sigma)$ is singular continuous if the covariance matrix $\Sigma \in \mathbb{R}^{d\times d}$ is singular. ($X$ is supported on a $\operatorname{rank}(\Sigma)$-dimensional affine subspace of $\mathbb{R}^d$.)

$\endgroup$
4
  • $\begingroup$ very interesting @Sangchul Lee. thank you. $\endgroup$
    – ashman
    Apr 1, 2022 at 19:23
  • $\begingroup$ Would you have any reference (book) about the singular part decomposition into $\mu_d$ and $\mu_{sc}$? $\endgroup$
    – Son Gohan
    Apr 2, 2022 at 12:33
  • 2
    $\begingroup$ @SonGohan Although I don't have a particular reference that immediately comes to my mind, but the decomposition of the singular part is fairly straightforward. You can first set $$\mu_{\text{d}} = \sum_{x} \mu(\{x\})\delta_x,$$ and then $$\mu_{\text{sc}} = \mu_{\text{sing}} - \mu_{\text{d}}.$$ $\endgroup$ Apr 2, 2022 at 12:43
  • $\begingroup$ I see, you basically remove the purely atomic part; what remains is a diffuse measure, still singular to Lebesgue one. Thanks! $\endgroup$
    – Son Gohan
    Apr 2, 2022 at 13:14
10
$\begingroup$

There is no definitive answer to the OP question. The examples given above are rather synthetic. For example, starting from the Lebesgue space $([0,1],\mathscr{B}([0,1]),m)$, where $m$ is the Lebesgue measure on $\mathscr{B}([0,1])$ the function $X(t)=\mathbb{1}_{[0,1/2]}$ has distribution $\beta(dx):=m\circ X^{-1}(dx)=\frac{1}{2}(\delta_0(dx)+\delta_1(dx))$ which is singular w.r.t. Lebesgue measure. This measure, called Bernoulli measure, happens to be the foundational measure in probability, that is, starting from the Bernoulli meansure $\beta$, and using the notion of independence, one can build Lebesgue measure and the whole theory of probability.

Similarly, one can manufacture all kinds of measures that singular w.r.t Lebesgue measure without any motivation behind it. On the other hand, interesting singular measures arise from specific applications in biology, physics, etc.

One nice example of a dynamical system is the humble logistic map $$ x_{n+1}:=f_\lambda(x_n)=\lambda x_n(1-x_n),\qquad x_0\in[0,1]$$ and $\lambda\in[0,4]$. A deep result by Misha Lyubich states that outside a set of measure zero $R$, for each $\lambda \in[0,4]\setminus R$, there is a unique invariant, and hence ergodic, measure $\mu_\lambda$ ( i.e. $\mu_\lambda f^{-1}=\mu_\lambda$) that is either absolutely continuous with respect the lebesgue measure $m$ on $[0,1]$, or supported in an attractive periodic orbit (hence it is singular w.r.t. Lebesgue measure).

It had previously been shown by Jakobson that the values of $\lambda\in R$ for which $\mu_\lambda$ has density has positive (Lebesgue) measure, and from the work of Yoccoz, this set is nowhere dense.

Lyubich, Graczyk and Swiatek that the open set of $\lambda$'s for which $\mu_\lambda$ is concentrated in periodic orbits is dense in $[0,4]$.

Even more striking is the behavior of the system where $\lambda^*\approx 3.449...$ (the Feigenbaum constant) where there is accumulation of period doubling, where the invariant measure is the devil's staircase distribution, which is concentrated in the ternary Cantor set. This measure $\mu_{\lambda^*}$ is not only a continuous measure ($\mu_{\lambda^*}(\{x\})=0$ for all $x\in[0,1]$) but singular with respect to Lebesgue’s.


There is a body of work by many people that study ergodic measures of dynamical systems, some systems modeling biological and/or physical systems, where behavior like that of the logistic map occurs.

$\endgroup$
7
$\begingroup$

Consider a random variable $X$ s.t. it takes either the value $0$ with probability $p$ or a continuously uniformly distributed value in $[-1,0)\cup(0,1]$ with probability $1-p$. This can model any IID measurement process in which we sometimes witness some uniform measurement error above or below $0$. Thus the 'density' is $$f(x)=p\delta(x)+\mathbf{1}_{[-1,0)\cup(0,1]}(x)\frac{(1-p)}{2}$$ and the distribution function is $$F_X(x)=\begin{cases}\frac{(1-p)(1+x)}{2}&-1\leq x <0\\ \frac{(1+p)+(1-p)x}{2}&0 \leq x \leq 1 \end{cases}$$ But you can notice that $f$ is no proper Lebesgue density (it has a Dirac delta) nor it can be a density wrt the counting measure.

$\endgroup$
6
  • 1
    $\begingroup$ The question specifically mentioned statistics, so imo this is the best answer so far. The considered statistical problem is realistic and often encountered in practice. $\endgroup$ Apr 2, 2022 at 13:23
  • $\begingroup$ @hasManyStupidQuestions thanks! Yes, I considered the 'statistical modelling' point of view of the OP. The other answers are on the 'applied mathematics' side. I think the whole thread is very interesting. $\endgroup$
    – Snoop
    Apr 2, 2022 at 13:59
  • $\begingroup$ Can you explain, in what scenario would you model a measurement this way, instead of just modelling it as uniform on $[-1,1]$? $\endgroup$
    – Jojo
    Apr 2, 2022 at 16:42
  • $\begingroup$ @Joe I have worked with high frequency data processes with 'sparse' deviations. In other words, a seemingly fixed (high) proportion would be 'exact' i.e. (zero, no deviation), while another proportion would display symmetric randomness around zero. Attempting modelling without taking this framework into account created super-peaked densities with no practical value. $\endgroup$
    – Snoop
    Apr 2, 2022 at 16:59
  • $\begingroup$ Oh that's very interesting and counterintuitive to me. Is this density really better at modelling your process than $f(x)=\mathbf{1}_{[-e,e]}(x)p + \mathbf{1}_{[-1,-e)\cup(e,1]}(x)\frac{(1-p)}{2}$ for some small $e$? $\endgroup$
    – Jojo
    Apr 2, 2022 at 17:25
5
$\begingroup$

As you mention you studied the Radon-Nikodym theorem, it is a good exercise if you haven't already seen the proof to try and prove the following useful fact.

Let $\nu$ stand for some fixed reference Borel measure on $\mathbb R^n$. For any Borel measure $\mu$ on $\mathbb R^n$, there exist Borel measures $\mu_{sing},\mu_{ac}$ such that $\mu = \mu_{sing} + \mu_{ac}$. Moreover, $\mu_{sing}$ is singular with respect to $\nu$, and $\mu_{ac}$ is absolutely continuous with respect to $\nu$. In applications of Fourier analysis to partial differential equations, considering the Fourier transform $\widehat{\mu}$ of certain measures is a powerful idea, and for linear PDE, the decomposition into singular and absolutely continuous parts (with respect to some reference measure) allows us to study $\widehat\mu_{sing}$ and $\widehat\mu_{ac}$ somewhat separately.

Remark: As an example, if $f$ is an absolutely continuous function, then $df/dx$ exists almost everywhere and defines a signed measure on $\mathbb R$. It's interesting to consider whether there are any criteria that tell us when $df/dx$ has zero singular part, and when $df/dx$ defines a positive measure.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .