3
$\begingroup$

I'm trying to sample a bivariate Gaussian distribution using Gibbs sampling, but I think I don't have the correct conditional probabilities. According to this lecture slides, the conditional expectation and variance of the bivariate Gaussian distribution are:

$E[X|Y=y]=\mu_X+\sigma_X\rho(\frac{\displaystyle y-\mu_Y}{\displaystyle \sigma_Y})$

and

$Var[X|Y=y]={{\sigma}_X}^2(1-{\rho}^2)$

So I'm sampling the bivariate Gaussian distribution using the standard normal distribution as follows:

$f(X|Y=y)=\mu_X+\sigma_X\rho(\frac{\displaystyle y-\mu_Y}{\displaystyle \sigma_Y})+{{\sigma}_X}^2(1-{\rho}^2)\mathcal{N}(0,1)$

$f(Y|X=x)=\mu_Y+\sigma_Y\rho(\frac{\displaystyle x-\mu_X}{\displaystyle \sigma_X})+{{\sigma}_Y}^2(1-{\rho}^2)\mathcal{N}(0,1)$

Are these equations ok to sample the bivariate Gaussian?

$\endgroup$
2
$\begingroup$

Conditionally on $[Y=y]$, $X$ is distributed like $$ \mu_X+\sigma_X\rho\sigma_Y^{-1}(y-\mu_Y)+\sigma_X\sqrt{1-\rho^2}Z, $$ where $Z$ is standard normal (note the factor $\sigma_X\sqrt{1-\rho^2}$ instead of its square). Likewise for $Y$ conditionally on $[X=x]$. But to sample $(X,Y)$, one should not use these two equations, rather, one can generate $Y$ by $$ Y=\mu_Y+\sigma_YT, $$ where $T$ is standard normal, then generate $X$ by $$ X=\mu_X+\sigma_X\rho\sigma_Y^{-1}(Y-\mu_Y)+\sigma_X\sqrt{1-\rho^2}Z, $$ where $Z$ is standard normal independent of $T$, or, equivalently, $$ X=\mu_X+\sigma_X\rho T+\sigma_X\sqrt{1-\rho^2}Z. $$

$\endgroup$
  • $\begingroup$ You are right about the equation, I should have used the factor instead of its square. And I think that you are also right about using only one equation for sampling, but since I'm using Gibbs sampling it seems ok to use both. Could you please check this example [link]?(pages.stat.wisc.edu/~mchung/teaching/stat471/lecture23.pdf) $\endgroup$ – Gorayni Jul 23 '13 at 5:38
  • $\begingroup$ One definitely do not use both, rather one uses each updating rule in succession--and a large number of times since the bivariate distribution one is interested in is the stationary distribution of the Markov chain, hence only at the limit does the process converge to it. $\endgroup$ – Did Jul 23 '13 at 8:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.