8
$\begingroup$

As far as I know orthogonality is a linear algebraic concept, where for a 2D or 3D case if the vectors are perpendicular we say they are orthogonal. Even it is OK for higher dimensions. But when it comes to random variables I cannot figure out orthogonality. I saw that somewhere if the expectation of 2 random variables $X$ and $Y$ is zero ( $E[XY] = 0$ ) then the random variables are orthogonal. How is that possible?

Is orthogonality in linear algebra and probability and statistics same?

$\endgroup$
6
$\begingroup$

Orthogonality comes from the idea of vanishing inner product. In case of random variables $$ \mathbb E \left [ X\right ] = \int_{-\infty}^\infty xd\mu_X $$ so, orthogonal RVs are those with $$ \mathbb E \left [ XY\right ] = \int_{-\infty}^\infty \int_{-\infty}^\infty xy d\mu_X d\mu_Y = 0 $$

$\endgroup$
  • $\begingroup$ What does dμX and dμY mean here? $\endgroup$ – dexterdev Aug 24 '13 at 7:02
  • 1
    $\begingroup$ $d\mu_X = f_X(x)dx$, where $f(x)$ - is probability density function. $\endgroup$ – Kaster Aug 24 '13 at 7:04
  • 7
    $\begingroup$ "Practically, it means that X and Y are independent"... Absolutely not. $\endgroup$ – Did Aug 24 '13 at 7:26
  • $\begingroup$ @Did, you're right. Deleted that line. $\endgroup$ – Kaster Aug 24 '13 at 7:31
  • $\begingroup$ @Did Do you mean statistically independent or loosely independent $\endgroup$ – dexterdev Aug 24 '13 at 8:24
6
$\begingroup$

Orthogonal means the vectors are at perpendicular to each other. We state that by saying that vectors x and y are orthogonal if their dot product (aka inner product) is zero, i.e. $x^\intercal y$=0.

However for vectors with random components, the orthogonality condition is modified to be Expected Value$E[x^\intercal y]=0$. This can be viewed as saying that for orthogonality, each random outcome of $x^\intercal y$ may not be zero, sometimes positive, sometimes negative, possibly also zero, but Expected Value $E[x^\intercal y]=0$. Keeping in mind, expected value is the same thing as the mean or average of possible outcomes.

Naturally when talking about orthogonality, we are talking about vectors.

$\endgroup$
1
$\begingroup$

The random variables $X$ and $Y$ can be thought of as vectors in a vector space (of infinite dimensions), equipped with an inner product, namely $\langle X, Y \rangle = \mathbb{E}[XY]$. This inner product defines a norm; for a random variable $X$, the norm is $\sqrt{\mathbb{E}[X^2]}$.

The inner product also satisfies symmetry, linearity and positivity as required of any inner product. We call two random variables $X$ and $Y$ as orthogonal when $\langle X, Y \rangle = \mathbb{E}[XY] = 0$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.