I am currently studying joint normal random variables and I came across the following fact:
Let $X$ and $Y$ be jointly normal random variables:
$$\begin{bmatrix} X \\Y \end{bmatrix} \sim N\left(\begin{bmatrix} 0 \\0 \end{bmatrix}, \begin{bmatrix} Var(X) & cov(X, Y) \\ cov(X, Y) & Var(Y) \end{bmatrix} \right)$$ where $Var(X)=1$, then $Y$ can be decomposed as $$Y = \gamma X + \eta $$ where $\gamma = cov(X, Y) = cov(X, Y)/Var(X)$ and $\eta$ is normal with mean zero and variance $Var(Y)-\gamma^2$ and is independent of $X$.
Why is the above true? Could someone show me a proof?
I can kind of see the intuition of where it comes from, for example, I know the $Y \mid X=x$ is normally distributed with mean $[cov(X,Y)/Var(X)] \cdot x = cov(X,Y)x$ and variance $Var(Y) - cov(X,Y)^2Var(X)^{-1} = Var(Y) - cov(X,Y)^2$. So using the decomposition and taking expectation conditional on $X$ gives $E(Y|X=x) = E(\gamma X|X=x) + E(\eta|X=x)=\gamma x = cov(X,Y)x$ since $\eta$ is independent of $X$. Similarly, $V(Y|X=x) = V(\eta|X=x) = V(\eta) = Var(Y)-\gamma^2$. So both the expectation and variance of the conditional distribution is correct.