I have troubles understanding a geometrical meaning of a covariance matrix.

Let's say we have a data set containing two points (-1,1), (-1,2) and write them in to the matrix

$$D = \begin{bmatrix} -1 & -1\\ 1 & 2 \end{bmatrix}. $$

Then we'll choose the unit-vector $v = \left( \frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}} \right)^T$.

And now, in this article is the following statement:

So, if we would like to represent the covariance matrix with a vector and its magnitude, we should simply try to find the vector that points into the direction of the largest spread of the data, and whose magnitude equals the spread (variance) in this direction.

If we define this vector as $v$, then the projection of our data $D$ onto this vector is obtained as $v^TD$, and the variance of the projected data is $v^T\Sigma v$.

The covariance matrix $\Sigma$ looks like this:

$$ \Sigma = \begin{bmatrix} 0 & 0\\ 0 & \frac{1}{2} \end{bmatrix} $$

Also, the projection of $D$ onto $v$ is the following vector:

$$ v^TD = \begin{bmatrix} 0\\ \sqrt{2} - \frac{1}{\sqrt{2}} \end{bmatrix} $$

So, in our case

$$ v^T\Sigma v = \left[ \frac{1}{\sqrt{2}}, \frac{1}{\sqrt{2}} \right] \begin{bmatrix} 0 & 0\\ 0 & \frac{1}{2} \end{bmatrix} \begin{bmatrix} \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}} \end{bmatrix} = \frac{1}{4}. $$

So, could you, please, explain, how can $v^T\Sigma v$ be a variance of values projected on $v$? I would really appreciate "the proof", that $v^T\Sigma v$ equals variance, as I don't see, why it is true.

  • 1
    $\begingroup$ The variance is $v^T\Sigma{v}$ not $v^TDv$ $\endgroup$ – David M. Mar 18 at 1:18
  • $\begingroup$ @DavidM. Thank you for the correction! $\endgroup$ – Eenoku Mar 18 at 1:45
  • 1
    $\begingroup$ Also, $v^TD\neq Dv$, as $D$ is not symmetric! $\endgroup$ – David M. Mar 18 at 1:49
  • $\begingroup$ @DavidM. Thank you again! $\endgroup$ – Eenoku Mar 18 at 1:59
  • 1
    $\begingroup$ @DavidM. They weren't pedantic, but necessary :) Something like $\begin{bmatrix} 0&0\\0&\frac{1}{2} \end{bmatrix}$ ? I'm afraid I exchanged columns with rows before in hurry... $\endgroup$ – Eenoku Mar 18 at 2:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.