Bivariate normal random variables decomposition

Question

I am currently studying joint normal random variables and I came across the following fact:

Let $X$ and $Y$ be jointly normal random variables:

$$\begin{bmatrix} X \\Y \end{bmatrix} \sim N\left(\begin{bmatrix} 0 \\0 \end{bmatrix}, \begin{bmatrix} Var(X) & cov(X, Y) \\ cov(X, Y) & Var(Y) \end{bmatrix} \right)$$ where $Var(X)=1$, then $Y$ can be decomposed as $$Y = \gamma X + \eta $$ where $\gamma = cov(X, Y) = cov(X, Y)/Var(X)$ and $\eta$ is normal with mean zero and variance $Var(Y)-\gamma^2$ and is independent of $X$.

Why is the above true? Could someone show me a proof?

I can kind of see the intuition of where it comes from, for example, I know the $Y \mid X=x$ is normally distributed with mean $[cov(X,Y)/Var(X)] \cdot x = cov(X,Y)x$ and variance $Var(Y) - cov(X,Y)^2Var(X)^{-1} = Var(Y) - cov(X,Y)^2$. So using the decomposition and taking expectation conditional on $X$ gives $E(Y|X=x) = E(\gamma X|X=x) + E(\eta|X=x)=\gamma x = cov(X,Y)x$ since $\eta$ is independent of $X$. Similarly, $V(Y|X=x) = V(\eta|X=x) = V(\eta) = Var(Y)-\gamma^2$. So both the expectation and variance of the conditional distribution is correct.

Answer 1

Use $Y = \gamma X + \eta$ to define $\eta := Y - \gamma X$. Now, we need to show two things:

1) $\eta$ has a normal distribution with mean $0$ and variance $Var(Y) - \gamma^2$

2) $\eta$ is independent of $X$

Make the transformation,

$$ \begin{pmatrix}\eta \\ X \end{pmatrix} = \underbrace{\begin{pmatrix}-\gamma & 1 \\ 1 & 0\end{pmatrix}}_A\begin{pmatrix}X \\ Y \end{pmatrix}$$

You should know that if $Z$ is a bivariate normal vector with mean vector $0$ and covariance matrix $\Sigma$, then $AZ$ is also bivariate normal with mean vector $0$ and covariance matrix, $A\Sigma A^T$. Use this fact to compute the covariance matrix of $\begin{pmatrix}\eta \\ X \end{pmatrix}$ above. You will see that the covariance matrix is diagonal, which implies independence for jointly normal vectors. Moreoever, the upper left diagonal element of the covariance matrix will be, $Var(Y) - \gamma^2$ as desired.