# On probabilistic independence and orthogonality

Originally asked in this post: Analogy of probabilistic independence with orthogonality, also available as Exercise $$24$$ of Tao's note: https://terrytao.wordpress.com/2015/10/12/275a-notes-2-product-measures-and-independence/comment-page-2/#comment-682682.

Let $${X}$$ be a random variable taking values in $${{\bf R}^n}$$ with the Gaussian distribution, in the sense that

$$\displaystyle \mathop{\bf P}( X \in S ) = \int_S \frac{1}{(2\pi)^{n/2}} e^{-|x|^2/2}\ dx$$

(where $${|x|}$$ denotes the Euclidean norm on $${{\bf R}^n})$$, and let $${v_1,\dots,v_m}$$ be vectors in $${{\bf R}^n}$$. Show that the random variables $${X \cdot v_1, \dots, X \cdot v_m}$$ (with $${\cdot}$$ denoting the Euclidean inner product) are jointly independent if and only if the $${v_1,\dots,v_m}$$ are pairwise orthogonal.

Question: Is there a more elementary approach to this problem by definition? e.g. Can we show directly that the random variables $${X \cdot v_1, \dots, X \cdot v_m}$$ are s.t $${\bf P}(\bigwedge_{i=1}^m (X \cdot v_i \in S_i)) = \prod_{i=1}^m {\bf P}(X \cdot v_i \in S_i)$$ for measurable subsets $$S_i \subset {\bf R}$$, if and only if the vectors $${v_1,\dots,v_m}$$ are pairwise orthogonal?

Note that we can omit any $$v_i$$ which are zero, since they're orthogonal to everything and $$X\cdot 0=0$$ the constant is independent of everything.

I'm going to use the following facts: An $$n\times n$$ matrix is orthogonal if its transpose is equal to its inverse. Such a matrix $$U$$ satisfies $$\|Ux\|=\|x\|=\|U^{-1}x\|$$ for all $$x$$ and $$|\det(U)|=1$$. Moreover, $$UX$$ has the same distribution as $$X$$ if $$X\sim N(0,I_n)$$ is the standard normal distribution in $$n$$ dimensions. We can directly deduce this from the pdf of $$X$$, since it depends only on $$\|x\|$$, which is preserved by $$U$$ (and $$|\det|(U)|=1$$ means the Jacobian of the change of variables $$u=Uv$$) is $$1$$.

If $$v_1,\ldots,v_m$$ are (non-zero and) orthogonal, then there exists an orthgonal $$U$$ such that $$Uv_i=\|v_i\|e_i$$ for $$1\leqslant i\leqslant m$$. Here $$e_1,\ldots,e_n$$ is the canonical basis of $$\mathbb{R}^n$$. Then $$Uv_1,\ldots,Uv_m$$ have the same joint distribution as $$\|v_1\|X_1,\ldots,\|v_m\|X_m$$, which are independent (because the pdf of $$X$$ splits into products of pdfs of its coordinates). This shows that orthogonality gives independence.

On the other hand, if they're not orthogonal, we can reduce to the $$n=2$$ case. if $$v_i,v_j$$ are not orthogonal for some $$i\neq j$$, there exist numbers $$a,b,c$$ and a unitary transformation $$U$$ such that $$Uv_i=ae_1$$ and $$Uv_j=be_1+ce_2$$ and $$a,b\neq 0$$. These vectors have the same joint distribution as $$aX_1$$ and $$bX_1+cX_2$$. We can look at the joint marginal distribution of $$X_1,X_2$$ (by integrating out $$X_3,\ldots, X_n$$) and assume $$n=2$$. Since we know $$aX_1,cX_2$$ are independent, and hence uncorrelated, it suffices to observe that $$aX_1,bX_1$$ are not uncorrelated, so that $$aX_1,bX_1+cX_2$$ are not uncorrelated, and therefore not independent.

• To make things a little more concrete, we can first try the model case where $v_1,\dots,v_m$ are the standard coordinate bases $e_1,\dots,e_m$, and set $m=n=2$. Professor Tao says that one can then compute the distributions of $X \cdot v_1, X \cdot v_2$ by direct computation (performing one-dimensional Gaussian integrals), and show that the product of these distributions agrees with the distribution of the joint random variable $X = (X \cdot v_1, X \cdot v_2)$. Commented Jan 2 at 22:30
• I don't quiet get the part of "compute the distribution of $X \cdot v_i$ by Gaussian integral". If $X$ is s.t$\displaystyle \mathop{\bf P}( X \in S ) = \int_S \frac{1}{(2\pi)^{n/2}} e^{-|x|^2/2}\ dx$, how do we know the distribution ${\bf P}(X \cdot v_i \in S)$ of the dot product to be a Gaussian integral? Commented Jan 2 at 22:37
• $$\mathbb{P}(UX\in S)=\mathbb{P}(X\in U^{-1}S)=\int_{U^{-1}S} f(x)dx \overset{y=Ux}{=}\int_Sf(U^{-1}y)|\det(U^{-1})|dy=\int_S f(y)dy=\int_S f(x)dx,$$ where $f$ is the $n$-dimensional $N(0,I)$ pdf. So $UX$,$X$ have the same distribution. $X\cdot v_i$ are the coordinates of $UX$, and the coordinates of a gaussian are gaussian.
– user1266745
Commented Jan 3 at 11:00