What does orthogonal random variables mean? As far as I know orthogonality is a linear algebraic concept, where for a 2D or 3D case if the vectors are perpendicular we say they are orthogonal. Even it is OK for higher dimensions. But when it comes to random variables I cannot figure out orthogonality. I saw that somewhere if the expectation of 2 random variables $X$ and $Y$ is zero (  $E[XY] = 0$  ) then the random variables are orthogonal. How is that possible?
Is orthogonality in linear algebra and probability and statistics same?
 A: Orthogonality comes from the idea of vanishing inner product. In case of random variables
$$
\mathbb E \left [ X\right ] = \int_{-\infty}^\infty xd\mu_X
$$
so, orthogonal RVs are those with
$$
\mathbb E \left [ XY\right ] = \int_{-\infty}^\infty \int_{-\infty}^\infty xy d\mu_X d\mu_Y = 0
$$
A: Orthogonal means the vectors are at perpendicular to each other. We state that by saying that vectors x and y are orthogonal if their dot product (aka inner product) is zero, i.e.
$x^\intercal y$=0.
However for vectors with random components, the orthogonality condition is modified to be
Expected Value$E[x^\intercal y]=0$. This can be viewed as saying that for orthogonality, each random outcome of $x^\intercal y$ may not be zero, sometimes positive, sometimes negative, possibly also zero, but Expected Value $E[x^\intercal y]=0$. Keeping in mind, expected value is the same thing as the mean or average of possible outcomes.
Naturally when talking about orthogonality, we are talking about vectors.
A: If $\langle X, Y \rangle$ = 0, then we say $X$ and $Y$ are orthogonal, where $X, Y$ are vectors in an inner product space with inner product $\langle \cdot, \cdot \rangle$.
Now, let $X, Y$ denote two random variables. Suppose $\langle X, Y \rangle = Cov(X,Y),$ where the latter denotes the covariance of $X$ and $Y.$ Then, one can verify that this is indeed an inner product (check the four properties of an inner product).
But, we also know that $Cov(X,Y) = \mathbb{E} [XY] - \mathbb{E} [X]\mathbb{E} [Y],$ so we have that
$$\langle X, Y \rangle = \mathbb{E} [XY] - \mathbb{E} [X]\mathbb{E} [Y].$$
If $X$ and $Y$ are independent, as the term is used in probability theory, then $\mathbb{E} [XY] = \mathbb{E} [X]\mathbb{E} [Y],$ so
$$\langle X, Y \rangle = \mathbb{E} [XY] - \mathbb{E} [X]\mathbb{E} [Y] = \mathbb{E} [X]\mathbb{E} [Y] - \mathbb{E} [X]\mathbb{E} [Y] = 0.$$ Therefore, $X$ and $Y$ are orthogonal, as the term is used in linear algebra.
A: The random variables $X$ and $Y$ can be thought of as vectors in a vector space (of infinite dimensions), equipped with an inner product, namely $\langle X, Y \rangle = \mathbb{E}[XY]$. This inner product defines a norm; for a random variable $X$, the norm is $\sqrt{\mathbb{E}[X^2]}$.
The inner product also satisfies symmetry, linearity and positivity as required of any inner product. We call two random variables $X$ and $Y$ as orthogonal when $\langle X, Y \rangle = \mathbb{E}[XY] = 0$.    
