14
$\begingroup$

As far as I know orthogonality is a linear algebraic concept, where for a 2D or 3D case if the vectors are perpendicular we say they are orthogonal. Even it is OK for higher dimensions. But when it comes to random variables I cannot figure out orthogonality. I saw that somewhere if the expectation of 2 random variables $X$ and $Y$ is zero ( $E[XY] = 0$ ) then the random variables are orthogonal. How is that possible?

Is orthogonality in linear algebra and probability and statistics same?

$\endgroup$
1

4 Answers 4

10
$\begingroup$

Orthogonal means the vectors are at perpendicular to each other. We state that by saying that vectors x and y are orthogonal if their dot product (aka inner product) is zero, i.e. $x^\intercal y$=0.

However for vectors with random components, the orthogonality condition is modified to be Expected Value$E[x^\intercal y]=0$. This can be viewed as saying that for orthogonality, each random outcome of $x^\intercal y$ may not be zero, sometimes positive, sometimes negative, possibly also zero, but Expected Value $E[x^\intercal y]=0$. Keeping in mind, expected value is the same thing as the mean or average of possible outcomes.

Naturally when talking about orthogonality, we are talking about vectors.

$\endgroup$
9
$\begingroup$

Orthogonality comes from the idea of vanishing inner product. In case of random variables $$ \mathbb E \left [ X\right ] = \int_{-\infty}^\infty xd\mu_X $$ so, orthogonal RVs are those with $$ \mathbb E \left [ XY\right ] = \int_{-\infty}^\infty \int_{-\infty}^\infty xy d\mu_X d\mu_Y = 0 $$

$\endgroup$
9
  • $\begingroup$ What does dμX and dμY mean here? $\endgroup$
    – dexterdev
    Aug 24, 2013 at 7:02
  • 1
    $\begingroup$ $d\mu_X = f_X(x)dx$, where $f(x)$ - is probability density function. $\endgroup$
    – Kaster
    Aug 24, 2013 at 7:04
  • 10
    $\begingroup$ "Practically, it means that X and Y are independent"... Absolutely not. $\endgroup$
    – Did
    Aug 24, 2013 at 7:26
  • $\begingroup$ @Did, you're right. Deleted that line. $\endgroup$
    – Kaster
    Aug 24, 2013 at 7:31
  • $\begingroup$ @Did Do you mean statistically independent or loosely independent $\endgroup$
    – dexterdev
    Aug 24, 2013 at 8:24
3
$\begingroup$

If $\langle X, Y \rangle$ = 0, then we say $X$ and $Y$ are orthogonal, where $X, Y$ are vectors in an inner product space with inner product $\langle \cdot, \cdot \rangle$.

Now, let $X, Y$ denote two random variables. Suppose $\langle X, Y \rangle = Cov(X,Y),$ where the latter denotes the covariance of $X$ and $Y.$ Then, one can verify that this is indeed an inner product (check the four properties of an inner product).

But, we also know that $Cov(X,Y) = \mathbb{E} [XY] - \mathbb{E} [X]\mathbb{E} [Y],$ so we have that $$\langle X, Y \rangle = \mathbb{E} [XY] - \mathbb{E} [X]\mathbb{E} [Y].$$ If $X$ and $Y$ are independent, as the term is used in probability theory, then $\mathbb{E} [XY] = \mathbb{E} [X]\mathbb{E} [Y],$ so $$\langle X, Y \rangle = \mathbb{E} [XY] - \mathbb{E} [X]\mathbb{E} [Y] = \mathbb{E} [X]\mathbb{E} [Y] - \mathbb{E} [X]\mathbb{E} [Y] = 0.$$ Therefore, $X$ and $Y$ are orthogonal, as the term is used in linear algebra.

$\endgroup$
1
$\begingroup$

The random variables $X$ and $Y$ can be thought of as vectors in a vector space (of infinite dimensions), equipped with an inner product, namely $\langle X, Y \rangle = \mathbb{E}[XY]$. This inner product defines a norm; for a random variable $X$, the norm is $\sqrt{\mathbb{E}[X^2]}$.

The inner product also satisfies symmetry, linearity and positivity as required of any inner product. We call two random variables $X$ and $Y$ as orthogonal when $\langle X, Y \rangle = \mathbb{E}[XY] = 0$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .