13
$\begingroup$

As far as I know orthogonality is a linear algebraic concept, where for a 2D or 3D case if the vectors are perpendicular we say they are orthogonal. Even it is OK for higher dimensions. But when it comes to random variables I cannot figure out orthogonality. I saw that somewhere if the expectation of 2 random variables $X$ and $Y$ is zero ( $E[XY] = 0$ ) then the random variables are orthogonal. How is that possible?

Is orthogonality in linear algebra and probability and statistics same?

$\endgroup$
1

4 Answers 4

8
$\begingroup$

Orthogonality comes from the idea of vanishing inner product. In case of random variables $$ \mathbb E \left [ X\right ] = \int_{-\infty}^\infty xd\mu_X $$ so, orthogonal RVs are those with $$ \mathbb E \left [ XY\right ] = \int_{-\infty}^\infty \int_{-\infty}^\infty xy d\mu_X d\mu_Y = 0 $$

$\endgroup$
9
  • $\begingroup$ What does dμX and dμY mean here? $\endgroup$
    – dexterdev
    Aug 24, 2013 at 7:02
  • 1
    $\begingroup$ $d\mu_X = f_X(x)dx$, where $f(x)$ - is probability density function. $\endgroup$
    – Kaster
    Aug 24, 2013 at 7:04
  • 8
    $\begingroup$ "Practically, it means that X and Y are independent"... Absolutely not. $\endgroup$
    – Did
    Aug 24, 2013 at 7:26
  • $\begingroup$ @Did, you're right. Deleted that line. $\endgroup$
    – Kaster
    Aug 24, 2013 at 7:31
  • $\begingroup$ @Did Do you mean statistically independent or loosely independent $\endgroup$
    – dexterdev
    Aug 24, 2013 at 8:24
8
$\begingroup$

Orthogonal means the vectors are at perpendicular to each other. We state that by saying that vectors x and y are orthogonal if their dot product (aka inner product) is zero, i.e. $x^\intercal y$=0.

However for vectors with random components, the orthogonality condition is modified to be Expected Value$E[x^\intercal y]=0$. This can be viewed as saying that for orthogonality, each random outcome of $x^\intercal y$ may not be zero, sometimes positive, sometimes negative, possibly also zero, but Expected Value $E[x^\intercal y]=0$. Keeping in mind, expected value is the same thing as the mean or average of possible outcomes.

Naturally when talking about orthogonality, we are talking about vectors.

$\endgroup$
1
$\begingroup$

The random variables $X$ and $Y$ can be thought of as vectors in a vector space (of infinite dimensions), equipped with an inner product, namely $\langle X, Y \rangle = \mathbb{E}[XY]$. This inner product defines a norm; for a random variable $X$, the norm is $\sqrt{\mathbb{E}[X^2]}$.

The inner product also satisfies symmetry, linearity and positivity as required of any inner product. We call two random variables $X$ and $Y$ as orthogonal when $\langle X, Y \rangle = \mathbb{E}[XY] = 0$.

$\endgroup$
1
$\begingroup$

If $\langle X, Y \rangle$ = 0, then we say $X$ and $Y$ are orthogonal, where $X, Y$ are vectors in an inner product space with inner product $\langle \cdot, \cdot \rangle$.

Now, let $X, Y$ denote two random variables. Suppose $\langle X, Y \rangle = Cov(X,Y),$ where the latter denotes the covariance of $X$ and $Y.$ Then, one can verify that this is indeed an inner product (check the four properties of an inner product).

But, we also know that $Cov(X,Y) = \mathbb{E} [XY] - \mathbb{E} [X]\mathbb{E} [Y],$ so we have that $$\langle X, Y \rangle = \mathbb{E} [XY] - \mathbb{E} [X]\mathbb{E} [Y].$$ If $X$ and $Y$ are independent, as the term is used in probability theory, then $\mathbb{E} [XY] = \mathbb{E} [X]\mathbb{E} [Y],$ so $$\langle X, Y \rangle = \mathbb{E} [XY] - \mathbb{E} [X]\mathbb{E} [Y] = \mathbb{E} [X]\mathbb{E} [Y] - \mathbb{E} [X]\mathbb{E} [Y] = 0.$$ Therefore, $X$ and $Y$ are orthogonal, as the term is used in linear algebra.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.