Find the unit vector so that this condition is true.

Question

Let $(X_1,X_2)$ be jointly normal with density

$$\phi(x_1,x_2;\rho) = \frac{1}{2\pi\sqrt{1-\rho^2}}\exp\left(\frac{-1}{2\sqrt{1-\rho^2}}(x_1^2 - 2\rho x_1x_2 + x_2^2)\right)$$

Find unit vector $a=(a_1,a_2)$ so that

$$Var(a_1 X_1 + a_2 X_2) \ge Var(b_1 X_1 + b_2 X_2)$$ for all unit vectors $b = (b_1,b_2)$.

After some algebra I'm at this point

$$\begin{pmatrix} a_1\\ a_2 \end{pmatrix}^T\begin{pmatrix} 1 & \rho\\ \rho & 1 \end{pmatrix} \begin{pmatrix} a_1\\ a_2 \end{pmatrix} \ge \begin{pmatrix} b_1\\ b_2 \end{pmatrix}^T\begin{pmatrix} 1 & \rho\\ \rho & 1 \end{pmatrix} \begin{pmatrix} b_1\\ b_2 \end{pmatrix} $$

But this has been all algebra to me. What is the geometric intuition here? And how do I proceed to find this $(a_1, a_2)$?

Answer 1

The variance of this random vector is $\begin{bmatrix} 1 & \rho \\ \rho & 1 \end{bmatrix}$.

Level sets of the density are ellipses, which are circles only when the correlation $\rho$ is $0$. With a positive-definite symmetric matrix, one can always rotate the two coordinate axes to diagonalize the matrix; then one has two linear combinations of the two random variables that are uncorrelated with each other. One of these will be in the direction of the major axis of the ellipse; the other the minor. You seek the former.

The variances of the two components are equal; therefore the distribution is invariant under interchanging the two variables $x_1$ and $x_2$, i.e. is invariant under reflection about the line $x_1=x_2$. The only non-circular ellipses centered at $(0,0)$ that are invariant under the interchange of $x_1$ and $x_2$ Have the lines $x_1=x_2$ and $x_1=-x_2$ as the two axes. Which is the major axis and which is the minor axis will be simply a matter of whether $\rho$ is positive or negative. So you have to rotate $45^\circ$. The unit vectors in the directions of the axes are $(1,1)/\sqrt{2}$ and $(-1,1)/\sqrt{2}$.

Notice also that the rotation matrix is $$ \begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \phantom{-}\cos\theta \end{bmatrix}. $$ So you can write the variance as a function of $\theta$: $$ \begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \phantom{-}\cos\theta \end{bmatrix}^T \begin{bmatrix} 1 & \rho \\ \rho & 1 \end{bmatrix} \begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \phantom{-}\cos\theta \end{bmatrix} $$ and see which value of $\theta$ makes this product a diagonal matrix.