Suppose the Gaussian random vector $\mathbf{X}\sim\mathcal{N}(\mathbf{\mu_X},\Sigma_\mathbf{X})$ where $$\mathbf{\mu_X}=\begin{bmatrix}1\\5\\2\end{bmatrix}$$ and $$\Sigma_\mathbf{X}=\begin{bmatrix}1&1&0\\1&4&0\\0&0&9\end{bmatrix}$$ I want to calculate the following PDFs: $$X_3\text{ given }\begin{bmatrix}X_1&X_2\end{bmatrix}^T$$ and $$X_1\text{ given }\begin{bmatrix}X_2&X_3\end{bmatrix}^T$$

For $X_3|\begin{bmatrix}X_1&X_2\end{bmatrix}^T$ we have that $X_3$, $X_1$ are uncorrelated since $\Sigma_\mathbf{X}(1,3)=cov(X_1,X_3)=0$ and thus independent as they are jointly Gaussian. Similarly $X_3$, $X_2$ are independent. So the conditional PDF becomes $$f_{X_3|\begin{bmatrix}X_1&X_2\end{bmatrix}^T}=f_{X_3}\sim\mathcal{N}(\mathbf{e_3^T}\mathbf{\mu_X},\mathbf{e_3^T}\Sigma_\mathbf{X}\mathbf{e_3})=\mathcal{N}(2,9)$$

for $\mathbf{e}_i$ a column vector with $1$ at position $i$ and $0$ elsewhere.

But in the other case of $X_1|\begin{bmatrix}X_2&X_3\end{bmatrix}^T$ even if $X_1$, $X_3$ are independent nevertheless $X_1$, $X_2$ are not and I cannot eliminate any of the r.v.s that appear nor can I imply conditional independence. Any ideas?


Since $(X_1,X_2)$ is independent of $X_3$, we have the following conditional PDF formula $$X_1|\{X_2=x_2,X_3=x_3\} = X_1|\{X_2=x_2\}\sim\mathcal{N}(\Sigma_{12}\Sigma_{22}^{-1}(x_2-\mu_2)+\mu_1, \Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21})$$

  • $\begingroup$ In your case, the conditional distribution of $X_1$ given $X_2$ and $X_3$ is the conditional distribution of $X_1$ given $X_2$ only. Did it become easier? $\endgroup$ – zhoraster Jan 14 '17 at 7:58
  • $\begingroup$ @zhoraster I am not sure that this is correct. Independence between two random variables does not necessarily imply conditional independence given a third random variable. $\endgroup$ – mgus Jan 14 '17 at 17:30
  • $\begingroup$ Here $X_3$ is independent of both $X_1$ and $X_2$. $\endgroup$ – zhoraster Jan 14 '17 at 19:26
  • $\begingroup$ @zhoraster OK, so I have found a formula for the conditional PDF of random vectors. I guess its proof deals with some tricks with the information and covariance representation of Gaussian random vectors. However, I am not sure how to calculate those $\Sigma_{ij}$ matrices. I couldn't find their definition. Can you check the edit above? $\endgroup$ – mgus Jan 14 '17 at 23:25
  • $\begingroup$ Well, here these matrices are $1\times 1$, so they're just numbers: $\Sigma_{11} = \Sigma_{12} = 1$, $\Sigma_{22} = 4$. $\endgroup$ – zhoraster Jan 15 '17 at 15:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.