# Correlation, Orthogonality, and Independence of pair of random variables in a Venn diagram

When I was studying the properties of pairs of random variables, I was introduced to the three (apparently) distinct concepts:

1. Orthogonality
2. Independence
3. Uncorrelatedness

I happen to realize that these three properties are somehow connected. After some sketching, I reached to following Venn diagram.

Given a pair of random variables, $$x, y \in \mathbb{R}$$, this Venn diagram aims to map how $$x$$ and $$y$$ can be related regarding uncorrelatedness, orthogonality, and independence.

It's important to mention that I didn't get this picture from any book¹. My goal here is to share the math analysis I used just to make sure it is nothing wrong. The six possibilities I found are:

1. $$x$$ and $$y$$ are neither uncorrelated, orthogonal, nor independent (case number 0).
2. $$x$$ and $$y$$ are uncorrelated, but not orthogonal or independent (case number 1)
3. $$x$$ and $$y$$ are independent (and consequently, uncorrelated), but they are not orthogonal (case number 2)
4. $$x$$ and $$y$$ are uncorrelated and orthogonal, but not independent (case number 3).
5. $$x$$ and $$y$$ are orthogonal, but not independent or uncorrelated (case number 4).
6. $$x$$ and $$y$$ are independent (and consequently, uncorrelated) and orthogonal (case number 5).

Futhermore, let me introduce $$\mu_x = E[x]$$ and $$\mu_y = \mathbb{E}[y]$$ as the mean of $$x$$ and $$y$$, respectively.

In example 6-30, Papoulis has stated that a pair of uncorrelated variables has covariance equals to zero, that is,

$$COV[x, y] = \sigma_{x, y} = E[(x - \mu_x)(y - \mu_y)] = 0 \tag{1}.$$

And Garcia states in section 5.6.2 that when the first joint moment of $$x$$ and $$y$$ is equal to 0, they are orthogonal, that is,

$$E[xy] = 0 \tag{2}.$$

### Case 0

Therefore, in the case 0, $$E[xy] \neq 0$$ and $$\sigma_{x, y} \neq 0$$.

### Case 4

In the case of number 4, we have that $$E[xy] = 0$$, but $$\sigma_{x, y} \neq 0$$. Note that

$$\sigma_{x, y} = E[xy - x\mu_y - y\mu_x + \mu_x\mu_y = E[xy] - \mu_x\mu_y \tag{3}.$$

Since $$E[xy]=0$$, we have that

$$\sigma_{x, y} = - \mu_x\mu_y \neq 0 \tag{4}.$$

Therefore, for the case 4, $$E[xy]=0$$ but $$x$$ and $$y$$ are nonzero-mean random variables as $$\mu_x \neq 0$$ and $$\mu_y \neq 0$$.

### Case 1

In the case 1, we have that $$\sigma_{x, y}=0$$ and $$E[xy] \neq 0$$. In this case, from the Equation $$(3)$$, we have that

$$\mu_x\mu_y = E[xy] \neq 0 \tag{5}.$$

Again, $$x$$ and $$y$$ are nonzero-mean random variables.

### Case 2

The case 2 deals with the independence of $$x$$ and $$y$$, which is a stronger statement than uncorrelatedness. Papoulis states in theorem 6.5 that, if

$$E[g(x)h(y)] = E[g(x)]E[h(y)]\;\;\; \forall\; g, h:\mathbb{R}\rightarrow\mathbb{R}, \tag{6}$$

then $$x$$ and $$y$$ are independent (and they belong to case number 2). Otherwise, they are merely uncorrelated (and they belong to case number 1).

### Case 3

For the case $$3$$, we have that $$\sigma_{x, y} = E[xy] = 0$$. From the equation $$3$$, that

$$\mu_x\mu_y = 0. \tag{7}$$

Hence, we can have three scenarios:

• $$\mu_x = 0$$ and $$\mu_y\neq 0$$.
• $$\mu_x \neq 0$$ and $$\mu_y = 0$$.
• $$\mu_x = \mu_y = 0$$.

Therefore, in the case 3, $$x$$ and $$y$$ are uncorrelated and orthogonal, and one of them or both are zero mean, but they are not independent.

When the are also independent, the equation $$(6)$$ is satisfied and they fall into the case 5.

Is this all correct?

¹: For probability and statistics, I use Leon Garcia and sometimes Papoulis, if you know any book that has a similar approach, please let me know.

PS: The areas depicted in the figure is not a set diagram. In other words, they do not represent intersection of random variables' sets. Rather, they represent what are the cases that a random variable may be classified regarding orthogonality, independence, uncorrelatedness. So their intersection area are merely illustrative.

• In the past I have also tried to visualize the independence through Venn Diagrams but is not easy, since independence require that $P(XY)=P(X)P(Y)$ which is not only related on how the sets are interconnected, but also in how are related the areas/weights, where defining a proportion of this kind is not easy in terms of Venn Diagrams: as example, see the figures shown in Wikipedia next to the section Pairwise and mutual independence. With this, first you need to give weights to the areas. Aug 18, 2022 at 0:26
• In $L^2$, independence of X and Y requires that the spaces of measurable functions generated by X and Y are orthogonal after removing their mean. Uncorrelatedness means that all linear functions of X and Y are orthogonal after removing their mean. Removing the mean is equivalent to an orthogonal projection onto the orthogonal complement of span{1}, where 1 is a representative of the equivalence class of random variables that take value 1 almost surely. Call this projection P. Then independence means P(f(X)) is orthogonal to P(g(Y)) for any measurable f,g. Uncorrelatedness is for linear f,g. Aug 18, 2022 at 0:44
• For a nice explanation of this, I like the first chapter of the time series book by Brockwell and Davis. Aug 18, 2022 at 0:50
• @Joako "but is not easy, since independence require that $P(XY)=P(X)P(Y)$". This is not a restriction for my analysis, at all. Keep in mind that my figure is not a set diagram, it is a Venn diagram that aims to understand all possibilities regarding uncorrelatedness, orthogonality, and independence. Their areas are merely representative and the equation $P(XY)=P(X)P(Y)$ is satisfied when $x$ and $y$ are independent or uncorrelated. Aug 18, 2022 at 12:11
• @gandalfbalrogslayer nice explanation! It could be inferred by my enunciate, since uncorrelated variables become orthorgonal after removing their means. Aug 18, 2022 at 12:18