# What is the Bernoulli product measure's Radon-Nikodym derivative wrt Lebesgue measure?

The Bernoulli product measure $$\mu$$ can be defined for each $$p\in (0,1)$$ on $$\Omega = \{0,1\}^\mathbb N=\{\omega=(\omega_i)|\omega_i\in\{0,1\}, i\in\mathbb N\}=\Pi_{i=1}^\infty \{0,1\}$$. The measure $$\mu$$ is essentially defined by a collection of Bernoulli random variables $$X_i:\Omega\to\{0,1\}$$ which take the value $$1$$ with probability $$p$$ and $$0$$ with probability $$1-p$$ depending only on the $$i$$th coordinate of $$\omega\in\Omega$$. The pre-images $$X_i^{-1}(0)$$ and $$X_i^{-1}(1)$$ are examples of cylinder sets in the product space and generate the $$\sigma$$-algebra that the product measure $$\mu$$ is defined on.

We can then define a map $$\Omega\to [0;1]\subset\mathbb R$$ by $$\omega=(\omega_i)\mapsto \sum_{i=1}^\infty \omega_i 2^{-i}$$. Up to a set of Lebesgue measure $$0$$ having to do with duplicate binary expansions of the dyadic rationals $$k/2^n$$ ($$k,n\in\mathbb N$$), this map identifies $$\Omega$$ with the unit interval $$[0,1]\subset\mathbb R$$. The duplicate issue is that dyadic rationals have both an infinite and a terminating binary expansion (in binary, $$1/2=0.1=0.0\bar 1$$), so the map is not a perfect bijection on that countable set.

We now have two measures on $$[0;1]$$, $$\mu$$ and Lebesgue measure $$\cal L$$. They are defined on slightly different $$\sigma$$-algebras since $$\cal L$$ is defined on the Borel sets and $$\mu$$ is defined on the $$\sigma$$-algebra generated by the sets that make the random variables $$X_i$$ measurable (generated by the cylinder sets).

Is there anything like a Radon-Nikodym derivative of $$\mu$$ with respect to $$\cal L$$ even though they are not defined on the same $$\sigma$$-algebras? For example, is it possible to extend or intersect the $$\sigma$$-algebras to obtain a useful integral formula with an actual density function so that we could write $$\mu=\nu + \int f d\cal L$$ on some meaningful collection of sets? ($$\nu$$ a measure singular wrt $$\cal L$$)

I've looked in a number of books including Billingsley, Chung, Folland, Rudin, Durrett, Cohn, and others. They discuss the construction of $$\mu$$, the differences of the $$\sigma$$-algebras that $$\mu$$ and $$\cal L$$ are defined on, but I haven't been able to find any kind of expression of one with respect to the other, if that's possible in some way.

My motivation is to get a better understanding of $$\mu$$ in more standard analytical terms.

PS: I originally posted this question over in

but it was recommended by a commenter that I post it here instead.

• +1 for the post and also for asking it here. Commented Jun 26, 2023 at 8:52
• For $p \neq 1/2,$ the measure induced by $\omega \to \sum \omega_i 2^{-i}$ on $[0,1]$ is w.r.t. the Lebesgue. This develops the idea in section 3 (as exercises, so not great if you're looking for a quick reference, but quite nice if you have the time to work through it). Commented Jun 27, 2023 at 7:08
• thanks @stochasticboy321 ! I'll work through the paper. on p.13 exercise 34 the paper references a random variable $W$ which doesn't have a density function when $p!=0.5$. I'm guessing that's related to FZan's Answer below. Commented Jun 27, 2023 at 15:49
• Oh whoops, I meant to say "singular w.r.t. the Lebesgue," kind of an important word to miss out :). But yes, it's very much related, and the ideas are fleshed out through the exercises in a quite pleasant way. Commented Jun 27, 2023 at 22:38

Let's first start by noting the measure $$\mu$$ is well defined on all Borel sets. Take

$$X : \Omega \longmapsto \mathbb{R}$$ $$X=\sum\limits_{i}\frac{X_{n}}{2^{n}}$$ Note that $$\sum\limits^{N}_{i}\frac{X_{i}}{2^{i}}$$ are random variables in $$\Omega$$ so its limit is in fact a random variable. We can then define a measure $$\mu_{X}$$ in all borel sets by setting $$\mu_{X}(B)=\mu(X^{-1}(B))$$. We can also get a little bit more intuition on how this measure works by looking on how it assigns measure to dyadic intervals.

Take any dyadic interval $$I=[\frac{b}{2^{n}},\frac{b+1}{2^{n}}]$$ of level $$n$$. By writing $$b$$ in base $$2$$ we may identify all dyadic intervals of level $$n$$ with strings of length $$n$$ in $$\{0,1\}^{[[n]]}$$. Now for any string $$\sigma$$ of length $$n$$ write $$I_{\sigma}$$ for the respective dyadic interval and $$C_{\sigma}$$ for the respective cylinder. We'll show by induction on $$n$$ that $$X^{-1}(I_{\sigma})=C_{\sigma}\cup D_{\sigma}$$ where $$D_{\sigma}$$ is a finite set (and thus of measure $$0$$ in $$\Omega$$).

The base case $$n=1$$ is clear if we notice that $$X \in [0,\frac{1}{2}]=I_{0}$$ if and only if $$X_{1}=0$$ or $$X_{1}=1$$ and $$X_{i}=0$$ for all $$i > 1$$, similarly for $$I_{1}=[\frac{1}{2},1]$$.

Now suppose we have proved our assertion for $$n>1$$. Take $$\sigma$$ any string of length of $$n+1$$. If $$\sigma=0\sigma'$$ for $$\sigma'$$ some string of length $$n$$, we have that $$X \in I_{\sigma}$$ if and only if $$2X \in I_{\sigma'}$$ and $$X_{1}=0$$. Write $$\Omega_{0}=\{\omega \in \Omega: X_{1}=0\}$$. The shift operator $$S(\omega)_{n}=\begin{cases} \omega_{0} & n=0 \\ \omega_{n+1} & \text{all other cases} \end{cases}$$

is a bijection from $$\Omega$$ to $$\Omega_{0}$$, furthermore, $$S(C_{\rho})=C_{0\rho}$$ for any string $$\rho$$. Now

$$2X|_{\Omega_{0}}=\sum\limits_{i}\frac{X_{i}}{2^{i-1}}=X \circ S^{-1}$$

So $$X^{-1}(I_{\sigma})=2X|_{\Omega_{0}}(I_{\sigma'})=(X \circ S^{-1})^{-1}(I_{\sigma'})=S(X(I_{\sigma'}))=S(C_{\sigma'}\cup D_{\sigma'})=C_{\sigma} \cup S(D_{\sigma'})$$

Here we used the inductive hypothesis on $$X^{-1}(I_{\sigma})$$. The case where $$\sigma=1\sigma'$$ can be done similarly by instead noting that $$X^{-1}(I_{\sigma})=(2X-1)|_{\Omega_{1}}^{-1}(I_{\sigma'})$$ where $$\Omega_{1}=\{\omega \in \Omega : X_{1} = 1\}$$. This completes the induction.

It is now clear that $$\mu_{X}(I_\sigma)=p^{\sum\sigma_{i}}(1-p)^{\sum (1-\sigma_{i})}$$

There is also a more "within $$\mathbb{R}$$" approach to finding the same measure through Carathéodory's extension theorem. By assigning this probability to each dyadic interval, we can construct exactly the same measure. Moreover in the special case where $$p=\frac{1}{2}$$, the uniqueness of the extension guarantees $$\mu_{X}=\mathcal{L}$$. Sadly that as far as we can go as for establishing relationships with $$\mathcal{L}$$.

For $$p$$ and $$q$$ write $$\mu_{p}$$ and $$\mu_{q}$$ for the measures constructed as we did above for Bernoulli random variables with parameters $$p$$ and $$q$$ respectively. We can prove that if $$p \neq q$$ then $$\mu_{p}$$ and $$\mu_{q}$$ are mutually singular, in particular if $$p \neq \frac{1}{2}$$, $$\mu_{p}$$ and $$\mathcal{L}$$ are mutually singular so Radon-Nikodym tells us little about $$\mu_{p}$$.

For a real number $$x$$ write $$x_{i}$$ for the $$i$$-th term in the binary expansion. By looking a carefully on how $$\mu_{X}$$ assigns probability to dyadic intervals, we can show that the functions $$f_{i}(x)=x_{i}$$ are independent Bernoulli random variables with parameter $$p$$ when looked at with respect to $$\mu_{p}$$ and of parameter $$q$$ with respect to the measure $$\mu_{q}$$. Thus, by the law of large numbers, the sets $$A_{p}=\{ x \in \mathbb{R} : \lim\limits_{n \to +\infty}\frac{\sum^{n}x_{i}}{n}=p\}$$ and $$A_{q}=\{ x \in \mathbb{R} : \lim\limits_{n \to +\infty}\frac{\sum^{n}x_{i}}{n}=q\}$$ fulfill

$$\mu_{p}(A_{p})=1$$ $$\mu_{q}(A_{q})=1$$

So $$\mu_{p}$$ and $$\mu_{q}$$ are mutually singular.

There is a little bit of discussion of the characteristics of these measures and of the sets $$A_{p}$$ from a fractal geometry perspective in first chapter of:

Bishop, Christopher J.; Peres, Yuval, Fractals in probability and analysis, Cambridge Studies in Advanced Mathematics 162. Cambridge: Cambridge University Press (ISBN 978-1-107-13411-9/hbk; 978-1-316-46023-8/ebook). ix, 402 p. (2017). ZBL1390.28012.

• Thanks @FZan ! I need to work through your response in more detail. The upshot seems to be that since $\mu$ ($p!=0$) and $\cal L$ are are mutually singular there is no integral component in the Radon-Nikodym representation. It's a little counter-intuitive to me since $\mu$ assigns positive measure to many intervals with interior, but then maybe that shouldn't surprise me. Commented Jun 27, 2023 at 15:52
• For those interested, there is an online version of the Bishop + Peres books: math.stonybrook.edu/~bishop/classes/math324.F15/book1Dec15.pdf Commented Jun 27, 2023 at 15:58
• I really appreciate this answer and I'm working through it. Here's where I'm stuck atm: if $\mu_X(I_\sigma)=p^{\sum\sigma_i}(1-p)^{\sum(1-\sigma_i)}>0$ and $\mathcal{L}(I_\sigma)>0$ (i.e., $\mathcal{L}$ being Lebesgue measure, aka $\mu_{1/2}$) how can those two measures be mutually singular? The definition of mutual singularity is "measures μ, ν are said to be mutually singular if there exists a measurable set s such that μ s = 0 and ν sᶜ = 0." so I would expect they can't both be positive on a Borel set with non-empty interior. Commented Jun 30, 2023 at 6:35
• "so I would expect they can't both be positive on a Borel set with non-empty interior. " This not necessarily the case, for example any two probability measures on $[0,1]$ must have $\mu([0,1])=\nu([0,1])=1$, mutually singular or not. They can have the same support even.
– FZan
Commented Jun 30, 2023 at 14:17
• In this case we have $\mu_{p}(A_{p}^c)=0$ and $A_{q}\subseteq A_{p}^c$ so $\mu_{q}(A_{p})=0$.
– FZan
Commented Jun 30, 2023 at 14:41