# What does orthogonal random variables mean?

As far as I know orthogonality is a linear algebraic concept, where for a 2D or 3D case if the vectors are perpendicular we say they are orthogonal. Even it is OK for higher dimensions. But when it comes to random variables I cannot figure out orthogonality. I saw that somewhere if the expectation of 2 random variables $X$ and $Y$ is zero ( $E[XY] = 0$ ) then the random variables are orthogonal. How is that possible?

Is orthogonality in linear algebra and probability and statistics same?

• Aug 24, 2013 at 6:34

Orthogonality comes from the idea of vanishing inner product. In case of random variables $$\mathbb E \left [ X\right ] = \int_{-\infty}^\infty xd\mu_X$$ so, orthogonal RVs are those with $$\mathbb E \left [ XY\right ] = \int_{-\infty}^\infty \int_{-\infty}^\infty xy d\mu_X d\mu_Y = 0$$

• What does dμX and dμY mean here? Aug 24, 2013 at 7:02
• $d\mu_X = f_X(x)dx$, where $f(x)$ - is probability density function. Aug 24, 2013 at 7:04
• "Practically, it means that X and Y are independent"... Absolutely not.
– Did
Aug 24, 2013 at 7:26
• @Did, you're right. Deleted that line. Aug 24, 2013 at 7:31
• @Did Do you mean statistically independent or loosely independent Aug 24, 2013 at 8:24

Orthogonal means the vectors are at perpendicular to each other. We state that by saying that vectors x and y are orthogonal if their dot product (aka inner product) is zero, i.e. $x^\intercal y$=0.

However for vectors with random components, the orthogonality condition is modified to be Expected Value$E[x^\intercal y]=0$. This can be viewed as saying that for orthogonality, each random outcome of $x^\intercal y$ may not be zero, sometimes positive, sometimes negative, possibly also zero, but Expected Value $E[x^\intercal y]=0$. Keeping in mind, expected value is the same thing as the mean or average of possible outcomes.

The random variables $$X$$ and $$Y$$ can be thought of as vectors in a vector space (of infinite dimensions), equipped with an inner product, namely $$\langle X, Y \rangle = \mathbb{E}[XY]$$. This inner product defines a norm; for a random variable $$X$$, the norm is $$\sqrt{\mathbb{E}[X^2]}$$.
The inner product also satisfies symmetry, linearity and positivity as required of any inner product. We call two random variables $$X$$ and $$Y$$ as orthogonal when $$\langle X, Y \rangle = \mathbb{E}[XY] = 0$$.
If $$\langle X, Y \rangle$$ = 0, then we say $$X$$ and $$Y$$ are orthogonal, where $$X, Y$$ are vectors in an inner product space with inner product $$\langle \cdot, \cdot \rangle$$.
Now, let $$X, Y$$ denote two random variables. Suppose $$\langle X, Y \rangle = Cov(X,Y),$$ where the latter denotes the covariance of $$X$$ and $$Y.$$ Then, one can verify that this is indeed an inner product (check the four properties of an inner product).
But, we also know that $$Cov(X,Y) = \mathbb{E} [XY] - \mathbb{E} [X]\mathbb{E} [Y],$$ so we have that $$\langle X, Y \rangle = \mathbb{E} [XY] - \mathbb{E} [X]\mathbb{E} [Y].$$ If $$X$$ and $$Y$$ are independent, as the term is used in probability theory, then $$\mathbb{E} [XY] = \mathbb{E} [X]\mathbb{E} [Y],$$ so $$\langle X, Y \rangle = \mathbb{E} [XY] - \mathbb{E} [X]\mathbb{E} [Y] = \mathbb{E} [X]\mathbb{E} [Y] - \mathbb{E} [X]\mathbb{E} [Y] = 0.$$ Therefore, $$X$$ and $$Y$$ are orthogonal, as the term is used in linear algebra.