I am a bit confused about the geometric interpretation of correlation as angle between two random variables. Suppose $X$ and $Y$ are two variables with mean $0$ and the state space $S=\{\omega_1, \omega_2,\omega_3\}$. Then $$Var(X)=X(\omega_1)^2\mathbb{P}(\omega_1)+X(\omega_2)^2\mathbb{P}(\omega_2)+X(\omega_3)^2\mathbb{P}(\omega_3)$$ $$Var(Y)=Y(\omega_1)^2\mathbb{P}(\omega_1)+Y(\omega_2)^2\mathbb{P}(\omega_2)+Y(\omega_3)^2\mathbb{P}(\omega_3)$$ $$Cov(X,Y)=X(\omega_1)Y(\omega_1)\mathbb{P}(\omega_1)+X(\omega_2)Y(\omega_2)\mathbb{P}(\omega_2)+X(\omega_3)Y(\omega_3)\mathbb{P}(\omega_3)$$ And the correlation $$\rho_{X,Y}=\frac{Cov(X,Y)}{\sqrt{Var(X)Var(Y)}}=\frac{X(\omega_1)Y(\omega_1)\mathbb{P}(\omega_1)+X(\omega_2)Y(\omega_2)\mathbb{P}(\omega_2)+X(\omega_3)Y(\omega_3)\mathbb{P}(\omega_3)}{\sqrt{(X(\omega_1)^2\mathbb{P}(\omega_1)+X(\omega_2)^2\mathbb{P}(\omega_2)+X(\omega_3)^2\mathbb{P}(\omega_3))(Y(\omega_1)^2\mathbb{P}(\omega_1)+Y(\omega_2)^2\mathbb{P}(\omega_2)+Y(\omega_3)^2\mathbb{P}(\omega_3))}}$$ I don't see how this is an angle between two vectors unless I define the two vectors $$x=[X(\omega_1)\sqrt{\mathbb{P}(\omega_1)}, X(\omega_2)\sqrt{\mathbb{P}(\omega_2)}, X(\omega_3)\sqrt{\mathbb{P}(\omega_3)}]$$ $$y=[Y(\omega_1)\sqrt{\mathbb{P}(\omega_1)}, Y(\omega_2)\sqrt{\mathbb{P}(\omega_2)}, Y(\omega_3)\sqrt{\mathbb{P}(\omega_3)}]$$ in which case I see that $$\rho_{X,Y}=\frac{<x,y>}{\sqrt{<x,x><y,y>}}=\cos\theta$$ where $\theta$ is the angle between $x$ and $y$. Is this the right way to interpret (as defining the vector of the value of each state weighted by the square root of the associate probability)?

  • $\begingroup$ The last quotient is not $\theta$ but $\cos\theta$ $\endgroup$
    – markvs
    Aug 17, 2020 at 3:18
  • $\begingroup$ The usual interpretation is as the cosine of the angle between the vectors, whence "cosine similarity" (in this case zero-centered). $\endgroup$
    – Brian Tung
    Aug 17, 2020 at 3:29

1 Answer 1


This is almost correct. To give such a geometric interpretation one needs to proceed exactly as you did and define two things:

  1. How is a random variable interpreted as a vector?
  2. How is the scalar product (or equivalently lengths and angles) between those vectors defined?

The interpretation for 1. is just the standard interpretation of functions as vectors. I.e. the random variables map the state space to $\mathbb{R}$ hence they are vectors such as every other real function. In your case the state space is finite hence the vector space is finite dimensional. You can identify it with $\mathbb{R}^3$ exactly as you suggested but you do not incorporate the probabilities! I.e. your random variable $X$ relates to the vector $(X(\omega_1), X(\omega_2), X(\omega_3)).$

The probabilities enter only for 2: Observe that the expectation of the product of zero mean random variables $\mathbb{E}[XY]$ fulfills all conditions of a scalar product this is bilinear, symmetric (pretty obviously) and nondegenerate since $\mathbb{E}[X^2]=0 \implies X=0$ a.e.

So you simply define $<X,Y>=\mathbb{E}[XY]$ and are ready to measure angles!


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .