14
$\begingroup$

This is Exercise 2.2.2 from Achim Klenke: »Probability Theory — A Comprehensive Course«.

Exercise (Box–Muller method): Let $U$ and $V$ be independent random variables that are uniformly distributed on $[0,1]$. Define $$X := \sqrt{−2\log(U)}\, \cos(2\pi V) \quad \text{and} \quad Y := \sqrt{−2\log(U)}\, \sin(2\pi V)\, .$$

Show that $X$ and $Y$ are independent and $\mathcal{N}_{0,1}$-distributed.

Solution: Define random variable $R:= \sqrt{-2\log(U)}$, then \begin{align*} \mathbf{P}\bigl[R \leq r\bigr] & = \mathbf{P}\bigl[-2 \log(U) \leq r^2\bigr] = \\ & = \mathbf{P}\bigl[\log(U) \geq -\frac{r^2}{2}\bigr] = \\ & = 1 - \mathbf{P}\biggl[U < \exp\Bigl(-\frac{r^2}{2}\Bigr)\biggr]\, . \end{align*} $U$ is uniformly defined on $[0, 1]$, so the distribution of $R$ is $$\mathbf{P}[R\leq r] = 1 - \int_0^{\exp(-r^2/2)} \, dt = 1 - \exp\Bigl(-\frac{r^2}{2}\Bigr)\, .$$ For the density of $R$ we get: $f_R(t) = \exp\Bigl(-\frac{t^2}{2}\Bigr)\cdot t$ with $t> 0$.

We also define the random variable $\Phi := 2\pi V$. Since $V$ is uniformly distributed on $[0, 1]$, $f_\Phi(t) = \frac{1}{2\pi}$ with $0< t \leq 2\pi$.

Since $U, V$ are independent, $R, \Phi$ must also be independent and $$f_{R, \Phi}(t_1, t_2) = f_R(t_1) f_\Phi(t_2) = \frac{1}{2 \pi} \exp\Bigl(-\frac{t_1^2}{2}\Bigr)\cdot t_1 \, .$$

With \begin{align*} g\colon (0,\infty)\times(0, 2\pi] &\rightarrow \mathbb{R}^2 \\ (r, \phi) &\mapsto \bigl(r \cos(\phi), r \sin(\phi)\bigr) \end{align*} we see that $$(X, Y) = g(R, \Phi)\, ,$$ so we want to find the image measure $$\mathbf{P}_{X, Y} = \mathbf{P}_{R, \Phi}\circ g^{-1}\, .$$

We use the transformation formula for densities: $$ f_{X, Y}(\tau_1, \tau_2) = \frac{f_{R, \Phi}(g^{-1}(\tau_1, \tau_2))}{|\det(g'(g^{-1}(\tau_1, \tau_2)))|}$$

$g$ is just the transformation for polar coordinates. With $$ t_1 = \sqrt{\tau_1^2 + \tau_2^2} = |\det(g'(g^{-1}(\tau_1, \tau_2)))|$$ we finally get $$f_{X, Y}(\tau_1, \tau_2) = \frac{1}{2 \pi} \exp\Bigl(-\frac{\tau_1^2 + \tau_2^2}{2}\Bigr) = \underbrace{\frac{1}{\sqrt{2 \pi}} \exp\Bigl(-\frac{\tau_1^2}{2}\Bigr)}_{=f_X(\tau_1)} \cdot \underbrace{\frac{1}{\sqrt{2 \pi}} \exp\Bigl(-\frac{\tau_2^2}{2}\Bigr)}_{=f_Y(\tau_2)}\, ,$$ that is: $X, Y$ are $\mathcal{N}_{0, 1}$-distributed and independent. $\square$


Could you please check my proof? I'm sorry that it's so long — it seems right to me, but I'm self-studying and really need to catch any eventual mistakes... Thank you!

$\endgroup$
  • $\begingroup$ Just today someone posted this question again. I think I answered it here several years ago. $\endgroup$ – Michael Hardy Jun 3 '15 at 20:28
  • $\begingroup$ I also asked this question yesterday. I solved it using the method I was attempting in my question. There was a little bit of algebra involved, but it actually wasn't too bad. I also found this link which discuss the problem in 2.4.3.mathematik.uni-ulm.de/numerik/teaching/ss09/NumFin/Script/… $\endgroup$ – user75514 Jun 4 '15 at 13:54
  • $\begingroup$ Looks about right to me. There's a bit of algebra involved in the transformation formula for densities (like the comment above mentions) that is skimmed over but the end result looks correct. $\endgroup$ – brian.keng Nov 23 '15 at 0:02
  • 2
    $\begingroup$ How do we know that the joint probability density factors out into probability densities of X and Y? Shouldn't we also have to calculate them directly? $\endgroup$ – Gytis Sep 22 '16 at 16:39
1
$\begingroup$

I am physicist and I can prove it faster (as a physicist ) intuitive derivation by using https://en.wikipedia.org/wiki/Inverse_transform_sampling

Imagine $2D$ Gaussian copula in polar coordinates $(\phi,r)$ representing $2D$ probability density in plane. (The copula is symmetric around the $Z-$axis.) $\rho(r)=A\exp(-\frac12r^2))$ where $A$ is a scaling parameter, such that $$\int_ 0^\infty 2\pi r \rho(r) dr = 1.$$(you can easily calculate the integral ; $2\pi r dr$ is area between two concentric close circles in $x,y$ plane) You get it from linear Gauss density function in Cartesian coordinates: $\rho(r)=\rho_1(x) \rho_1(y)$ where $\rho_1(x)=\sqrt{A}\exp(-\frac12x^2))$ (using Pythagoras theorem or euclidean metric as well)

Let's generate random $3D$ points uniformly under the the Gauss copula. Projecting the uniformly distributed samples in volume $(X,Y,Z)$ or $(r,\phi,z)$ into the base plane $(X,Y)$ or $(r,\phi)$ provides the Gaussian distributed samples in the $2D$ plane. Projecting those samples into any direction (for example $X$ or $Y$ axis) we get the chosen Gaussian distribution.

There is a construction you can split the whole volume under the Gaussian copula into small objects of the same volume of $dV=drd\rho dh$ and attach an index label to them from $1$ into $N$, where $N$ is around $\frac{1}{dV}$. $dr$ is radial size, $d\rho$ is tangential size (perpendicular to $dr$) and $dh$ is height in $Z$ axis. Now you can generate uniformly numbers $i=1 \ldots N$ (or from interval $\rho=[0 \ldots 1]$ multiplied by $N$ to pick by random uniformly the volume $dV_i$. This is uniform volume sampling. For purists to get true random points sampling you can combine it with random triplet $(r_1,\rho_1,h1)$ to pick random point inside random $dV_i$. We proved that uniformly volume sampling leads to Gaussian distribution of the projection of its samples. We chose any indexing system of $dV$ with the only constraint: Let $dV_i$ has $r_i$ coordinate and $dV_j$ has $r_j$ coordinate. If $r_i<r_j$ then $i<j$. Indexing is growing with $r$. We leave the indexing of within the wall unspecified on purpose.

We will generate uniformly a real number $P$ from $(0,1)$ and when multiplied by $N$ that will pick a volume $dV_i$ somewhere inside a cylinder wall (of $dr$ thickness) around $Z$ with radius $R$ calculated as: Probability or fraction $P$ of uniformly distributed random points under copula with $r$ closer than $R$ (inside the cylinder) is integral from $0$ to $R \rho(r) 2 \pi r.dr$. We can get easily (this is the reason why we work in polar coordinates) a closed formula of this integral by simple substitution $\frac12 r^2=p$ and $rdr=dp$

\begin{align} P&=A (2\pi \int_0^{\frac12 R^2} \exp(-p) dp \\ &=A (2 \pi \left[-\exp(-p)\right]_0^{\frac12 R^2} \\ &=A (2\pi)[1-\exp(-\frac12 R^2)] \end{align}

Notice that when $R$ goes to $\infty$ we normalize the probability the integral goes to $1$ so we have $A=1/(2 \pi)$ ).
We solve for $R$:

$$R(P)=\sqrt{-2\ln(1-P)} $$

That is:An uniformly generated $P$ produces a correct $R$. We know we should get uniformly distributed $\phi$ coordinate as well. One index $i$ generated above should provide this uniformly distributed $\phi$ (due to symmetry) as well if you have defined an volume indexing/labeling system within the cylinder wall, but we neglected the details of indexing system to get $\phi$ easily now. We simply generate uniformly distributed $\phi \in (0,2\pi)$ now. At the end we do projection by polar to Cartesian coordinates transformation to get the celebrated Gaussian distribution of random samples.

$$x=R\cos(\phi) $$ $$y=R\sin(\phi)$$

In formula for $R(P)$ above notice that if $P$ is uniformly distributed on $(0,1)$, then $1-P$ is as well. You can replace $1-P$ with $P$ in the method.

$$R(P)=\sqrt{-2\ln(P)} $$

$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.