What does orthogonal random variables mean?

As far as I know orthogonality is a linear algebraic concept, where for a 2D or 3D case if the vectors are perpendicular we say they are orthogonal. Even it is OK for higher dimensions. But when it comes to random variables I cannot figure out orthogonality. I saw that somewhere if the expectation of 2 random variables $X$ and $Y$ is zero ( $E[XY] = 0$ ) then the random variables are orthogonal. How is that possible?

Is orthogonality in linear algebra and probability and statistics same?

• Aug 24, 2013 at 6:34

Orthogonal means the vectors are at perpendicular to each other. We state that by saying that vectors x and y are orthogonal if their dot product (aka inner product) is zero, i.e. $x^\intercal y$=0.

However for vectors with random components, the orthogonality condition is modified to be Expected Value$E[x^\intercal y]=0$. This can be viewed as saying that for orthogonality, each random outcome of $x^\intercal y$ may not be zero, sometimes positive, sometimes negative, possibly also zero, but Expected Value $E[x^\intercal y]=0$. Keeping in mind, expected value is the same thing as the mean or average of possible outcomes.

Naturally when talking about orthogonality, we are talking about vectors.

Orthogonality comes from the idea of vanishing inner product. In case of random variables $$\mathbb E \left [ X\right ] = \int_{-\infty}^\infty xd\mu_X$$ so, orthogonal RVs are those with $$\mathbb E \left [ XY\right ] = \int_{-\infty}^\infty \int_{-\infty}^\infty xy d\mu_X d\mu_Y = 0$$

• What does dμX and dμY mean here? Aug 24, 2013 at 7:02
• $d\mu_X = f_X(x)dx$, where $f(x)$ - is probability density function. Aug 24, 2013 at 7:04
• "Practically, it means that X and Y are independent"... Absolutely not.
– Did
Aug 24, 2013 at 7:26
• @Did, you're right. Deleted that line. Aug 24, 2013 at 7:31
• @Did Do you mean statistically independent or loosely independent Aug 24, 2013 at 8:24

If $$\langle X, Y \rangle$$ = 0, then we say $$X$$ and $$Y$$ are orthogonal, where $$X, Y$$ are vectors in an inner product space with inner product $$\langle \cdot, \cdot \rangle$$.

Now, let $$X, Y$$ denote two random variables. Suppose $$\langle X, Y \rangle = Cov(X,Y),$$ where the latter denotes the covariance of $$X$$ and $$Y.$$ Then, one can verify that this is indeed an inner product (check the four properties of an inner product).

But, we also know that $$Cov(X,Y) = \mathbb{E} [XY] - \mathbb{E} [X]\mathbb{E} [Y],$$ so we have that $$\langle X, Y \rangle = \mathbb{E} [XY] - \mathbb{E} [X]\mathbb{E} [Y].$$ If $$X$$ and $$Y$$ are independent, as the term is used in probability theory, then $$\mathbb{E} [XY] = \mathbb{E} [X]\mathbb{E} [Y],$$ so $$\langle X, Y \rangle = \mathbb{E} [XY] - \mathbb{E} [X]\mathbb{E} [Y] = \mathbb{E} [X]\mathbb{E} [Y] - \mathbb{E} [X]\mathbb{E} [Y] = 0.$$ Therefore, $$X$$ and $$Y$$ are orthogonal, as the term is used in linear algebra.

The random variables $$X$$ and $$Y$$ can be thought of as vectors in a vector space (of infinite dimensions), equipped with an inner product, namely $$\langle X, Y \rangle = \mathbb{E}[XY]$$. This inner product defines a norm; for a random variable $$X$$, the norm is $$\sqrt{\mathbb{E}[X^2]}$$.

The inner product also satisfies symmetry, linearity and positivity as required of any inner product. We call two random variables $$X$$ and $$Y$$ as orthogonal when $$\langle X, Y \rangle = \mathbb{E}[XY] = 0$$.