# Conditional distribution of Normal distribution with singular covariance

I'm stuck with the following problem: Suppose you have a n-variate random vector like $$Y = S + \epsilon,$$ $$S = LZ$$ where $\epsilon \sim N_n(0,I\sigma^2)$, $L$ is a known singular matrix, and $Z \sim N_n(0,I)$ ($N_n$ stands for an n-variate Normal distribution). In this case, the values of $S$ belongs to a space with dimension less than $n$, but since there's an error term $\epsilon$, $Y$ goes back to $\mathbb R^n$. How can I find the conditional distribution of $S|Y$? The fact that $LL'$ has no inverse makes the usual method fail because $S$ has no pdf.

As an example, suppose $n=2$, and $L =\bigl( \begin{smallmatrix}1 & 0\\1 & 0 \end{smallmatrix}\bigr)$, then $S$ must be in the $x=y$ line. My intuition says $S|Y$ that it has to be Normal Distributed, with mean $\frac{Y_1 + Y_2}{2}$ and applying the transformation $L$ to something. But I can't figure out the math behind this.

The "usual formula" that I know requires only that the covariance matrix of $Y$, namely $K:=LL'+\sigma^2I$, is invertible. Writing $M:=LL'$, you have that the conditional distribution of $S$, given that $Y=y$, is $n$-variate normal with mean vector $MK^{-1}y$ and covariance matrix $M-MK^{-1}M$. In your example $M=\left( \begin{smallmatrix}1 & 1\\1 & 1 \end{smallmatrix}\right)$ and $K=\left( \begin{smallmatrix}1+\sigma^2 & 1\\1 & 1+\sigma^2 \end{smallmatrix}\right)$, so the conditional mean vector is $\left( \begin{smallmatrix}(y_1+y_2)/(2+\sigma^2)\\(y_1+y_2)/(2+\sigma^2) \end{smallmatrix}\right)$ and the conditional covariance is $\left( \begin{smallmatrix}\sigma^2/(2+\sigma^2) & \sigma^2/(2+\sigma^2)\\\sigma^2/(2+\sigma^2) & \sigma^2/(2+\sigma^2) \end{smallmatrix}\right)$. (I am assuming that $\epsilon$ and $Z$ are independent.)
• What I call "usual method" is using Bayes theorem: $f_{S|Y} = \frac{f_{Y|S} f_S}{f_Y}$, but in this case, $f_S$ involves $(LL')^{-1}$. Can you give me a brief description of the steps involved? Jul 7, 2017 at 19:53
• Suppose you have an $n\times n$ matrix $A$ such that $S-AY$ and $AY$ are independent. Then the conditional distribution of $S=(S-AY)+AY$, given that $Y=y$, is the same as $(S-AY)+Ay$. The vector $S-AY$ is normal with mean vector $0$ and covariance $M-AKA'$. By computing covariances (after all, for joint normal random variables, zero covariance implies independence) you see that $A=MK^{-1}$ does the trick. Jul 7, 2017 at 23:07