# Covariance, inner product and independence

Let $X$ and $Y$ be two normally distributed random variables.

I understand that if $\operatorname{cov}(X,Y)=0$ then they are stochastically independent (right?).

Now, is it true that if $X \cdot Y = 0$ they are independent?, are those two results equivalent?

Note that there are two ways in which $X$ and $Y$ can be normally distributed:

• Way A. When each is normally distributed, i.e. $X \sim \mathcal N(\mu_X, \sigma^2_X)$ and $Y \sim \mathcal N(\mu_Y, \sigma^2_Y)$.

• Way B. When $X$ and $Y$ are jointly normally distributed. In this case, one could think of a new random variable $(X,Y) \sim \mathcal N(\mu, \Sigma)$. For definition see here.

As is immediately apparent from the definition, Way A is strictly weaker than Way B. So when someone tells you

Let $X$ and $Y$ be two normally distributed random variables.

you should assume (since the keywords "jointly" or "multivariate normal" are missing) that they only mean Way A.

And assuming Way A, the following

if $\operatorname{cov}(X,Y)=0$ then they are stochastically independent

does not hold. For counterexamples and some pictures see this. For more pictures and intuition see this.

By the way, assuming Way B we do have

if $\operatorname{cov}(X,Y)=0$ then they are stochastically independent.

To prove it, show that for any real $x$ and $y$ holds $f_{(X,Y)}(x,y) = f_X(x)f_Y(y)$ (where $f_{(X,Y)}$ is the density of $(X,Y)$ a.k.a. the joint density of $X$ and $Y$).

Now to the second part of the question:

Now, is it true that if $X \cdot Y = 0$ they are independent?

First of all, what is $X\cdot Y$? Well, I see a dot there, is this a dot product? Since the things you plug into it are random variables, I think (see dot vs inner product) that you meant it to be an inner product.

And an inner product (on a vector space $V$ over a field $F$) is just any map \begin{align} \langle \cdot , \cdot \rangle: V\times V \to F \end{align} that satisfies certain properties.

Nothing stops us from defining the following map: \begin{align} \operatorname{cov} : L^2 \times L^2 & \to \mathbb R\\ (X,Y) & \mapsto \mathbb E \Big( \big[(X - \mathbb E[X])(Y - \mathbb E[Y])\big] \Big) \end{align}

where you can roughly think of $L^2$ as the vector space of all random variables (on some underlying probability space) that have finite variance.

As it turns out, the map $\operatorname{cov}$ does indeed satisfy the properties to call itself an inner product.

So in the context of random variables, if someone writes $X\cdot Y$ it is (unless clearly noted otherwise) by definition the same as $\operatorname{cov}(X,Y)$.

• A bit of nitpicking, $\operatorname{cov}$ is not a inner product on all of $L^2(\mathbb{P})$ as it is degenerate. Of course this deficiency can be easily avoided either taking quotient or focusing on an appropriate subspace. And (+1) Sep 21, 2017 at 14:42
• @SangchulLee: Thanks! You're right, of course, and that's why I wrote "you can roughly think of $L^2$ as ..." since I decided not to go into the difference between $\mathcal L^2$ (all r.v. with finite variance) and $L^2$ (the quotient with "almost surely equal" equivalence classes).
– ruh
Sep 21, 2017 at 15:12
• Aha, I was certain that you are also aware of this fact, so I was wondering why you just wrote 'inner product'. That explains my question. Thank you! :) Sep 21, 2017 at 15:13
• Further nitpicking concerning the "almost surely equal" comment - it should be "equal up to an additive constant (almost surely)", since cov doesn't "see" additive constants, see also en.wikipedia.org/wiki/Covariance#Relationship_to_inner_products Sep 2, 2019 at 7:35
• Hello @adam, why it is $L^2$ but not $L$? What does the 2 mean? Nov 19, 2019 at 5:57