When I was studying the properties of pairs of random variables, I was introduced to the three (apparently) distinct concepts:

  1. Orthogonality
  2. Independence
  3. Uncorrelatedness

I happen to realize that these three properties are somehow connected. After some sketching, I reached to following Venn diagram.

enter image description here

Given a pair of random variables, $x, y \in \mathbb{R}$, this Venn diagram aims to map how $x$ and $y$ can be related regarding uncorrelatedness, orthogonality, and independence.

It's important to mention that I didn't get this picture from any book¹. My goal here is to share the math analysis I used just to make sure it is nothing wrong. The six possibilities I found are:

  1. $x$ and $y$ are neither uncorrelated, orthogonal, nor independent (case number 0).
  2. $x$ and $y$ are uncorrelated, but not orthogonal or independent (case number 1)
  3. $x$ and $y$ are independent (and consequently, uncorrelated), but they are not orthogonal (case number 2)
  4. $x$ and $y$ are uncorrelated and orthogonal, but not independent (case number 3).
  5. $x$ and $y$ are orthogonal, but not independent or uncorrelated (case number 4).
  6. $x$ and $y$ are independent (and consequently, uncorrelated) and orthogonal (case number 5).

Futhermore, let me introduce $\mu_x = E[x]$ and $\mu_y = \mathbb{E}[y]$ as the mean of $x$ and $y$, respectively.

In example 6-30, Papoulis has stated that a pair of uncorrelated variables has covariance equals to zero, that is,

$$COV[x, y] = \sigma_{x, y} = E[(x - \mu_x)(y - \mu_y)] = 0 \tag{1}.$$

And Garcia states in section 5.6.2 that when the first joint moment of $x$ and $y$ is equal to 0, they are orthogonal, that is,

$$ E[xy] = 0 \tag{2}.$$

Case 0

Therefore, in the case 0, $ E[xy] \neq 0 $ and $\sigma_{x, y} \neq 0$.

Case 4

In the case of number 4, we have that $E[xy] = 0$, but $\sigma_{x, y} \neq 0$. Note that

$$ \sigma_{x, y} = E[xy - x\mu_y - y\mu_x + \mu_x\mu_y = E[xy] - \mu_x\mu_y \tag{3}.$$

Since $E[xy]=0$, we have that

$$\sigma_{x, y} = - \mu_x\mu_y \neq 0 \tag{4}.$$

Therefore, for the case 4, $E[xy]=0$ but $x$ and $y$ are nonzero-mean random variables as $\mu_x \neq 0$ and $\mu_y \neq 0$.

Case 1

In the case 1, we have that $\sigma_{x, y}=0$ and $E[xy] \neq 0$. In this case, from the Equation $(3)$, we have that

$$ \mu_x\mu_y = E[xy] \neq 0 \tag{5}.$$

Again, $x$ and $y$ are nonzero-mean random variables.

Case 2

The case 2 deals with the independence of $x$ and $y$, which is a stronger statement than uncorrelatedness. Papoulis states in theorem 6.5 that, if

$$ E[g(x)h(y)] = E[g(x)]E[h(y)]\;\;\; \forall\; g, h:\mathbb{R}\rightarrow\mathbb{R}, \tag{6}$$

then $x$ and $y$ are independent (and they belong to case number 2). Otherwise, they are merely uncorrelated (and they belong to case number 1).

Case 3

For the case $3$, we have that $\sigma_{x, y} = E[xy] = 0$. From the equation $3$, that

$$\mu_x\mu_y = 0. \tag{7}$$

Hence, we can have three scenarios:

  • $\mu_x = 0$ and $\mu_y\neq 0$.
  • $\mu_x \neq 0$ and $\mu_y = 0$.
  • $\mu_x = \mu_y = 0$.

Therefore, in the case 3, $x$ and $y$ are uncorrelated and orthogonal, and one of them or both are zero mean, but they are not independent.

When the are also independent, the equation $(6)$ is satisfied and they fall into the case 5.

Is this all correct?

¹: For probability and statistics, I use Leon Garcia and sometimes Papoulis, if you know any book that has a similar approach, please let me know.

PS: The areas depicted in the figure is not a set diagram. In other words, they do not represent intersection of random variables' sets. Rather, they represent what are the cases that a random variable may be classified regarding orthogonality, independence, uncorrelatedness. So their intersection area are merely illustrative.

  • $\begingroup$ In the past I have also tried to visualize the independence through Venn Diagrams but is not easy, since independence require that $P(XY)=P(X)P(Y)$ which is not only related on how the sets are interconnected, but also in how are related the areas/weights, where defining a proportion of this kind is not easy in terms of Venn Diagrams: as example, see the figures shown in Wikipedia next to the section Pairwise and mutual independence. With this, first you need to give weights to the areas. $\endgroup$
    – Joako
    Aug 18, 2022 at 0:26
  • $\begingroup$ In $L^2$, independence of X and Y requires that the spaces of measurable functions generated by X and Y are orthogonal after removing their mean. Uncorrelatedness means that all linear functions of X and Y are orthogonal after removing their mean. Removing the mean is equivalent to an orthogonal projection onto the orthogonal complement of span{1}, where 1 is a representative of the equivalence class of random variables that take value 1 almost surely. Call this projection P. Then independence means P(f(X)) is orthogonal to P(g(Y)) for any measurable f,g. Uncorrelatedness is for linear f,g. $\endgroup$ Aug 18, 2022 at 0:44
  • $\begingroup$ For a nice explanation of this, I like the first chapter of the time series book by Brockwell and Davis. $\endgroup$ Aug 18, 2022 at 0:50
  • $\begingroup$ @Joako "but is not easy, since independence require that $P(XY)=P(X)P(Y)$". This is not a restriction for my analysis, at all. Keep in mind that my figure is not a set diagram, it is a Venn diagram that aims to understand all possibilities regarding uncorrelatedness, orthogonality, and independence. Their areas are merely representative and the equation $P(XY)=P(X)P(Y)$ is satisfied when $x$ and $y$ are independent or uncorrelated. $\endgroup$ Aug 18, 2022 at 12:11
  • $\begingroup$ @gandalfbalrogslayer nice explanation! It could be inferred by my enunciate, since uncorrelated variables become orthorgonal after removing their means. $\endgroup$ Aug 18, 2022 at 12:18


You must log in to answer this question.