I know that if $X$ and $Y$ are random variables with respective PDFs,

$$ f_X(x) = \frac{1}{\sqrt{2\pi\sigma_x^2}}\exp\left\{-\frac{\left(x-\mu_x\right)^2}{2\sigma_x^2}\right\} \\ f_Y(y) = \frac{1}{\sqrt{2\pi\sigma_y^2}}\exp\left\{-\frac{\left(y-\mu_y\right)^2}{2\sigma_y^2}\right\} $$

Then their joint PDF is written as

$$ f_{XY}(x,y) = \frac{1}{2 \pi \sigma_x \sigma_y \sqrt{1-\rho^2}} \exp\left( -\frac{1}{2(1-\rho^2)}\left[ \frac{(x-\mu_x)^2}{\sigma_x^2} - \frac{2\rho(x-\mu_x)(y-\mu_y)}{\sigma_x \sigma_y} + \frac{(y-\mu_y)^2}{\sigma_y^2} \right] \right) $$

But when $\mathbf{x}$ and $\mathbf{y}$ are random vectors with PDFs

$$ f_{\mathbf x}(x_1,\ldots,x_k)\, = \frac{1}{(2\pi)^{k/2}|\boldsymbol\Sigma_x|^{1/2}} \exp\left(-\frac{1}{2}({\mathbf x}-{\boldsymbol\mu_x})^T{\boldsymbol\Sigma_x}^{-1}({\mathbf x}-{\boldsymbol\mu_x}) \right) \\ f_{\mathbf y}(y_1,\ldots,y_k)\, = \frac{1}{(2\pi)^{k/2}|\boldsymbol\Sigma_y|^{1/2}} \exp\left(-\frac{1}{2}({\mathbf y}-{\boldsymbol\mu_y})^T{\boldsymbol\Sigma_y}^{-1}({\mathbf y}-{\boldsymbol\mu_y}) \right) $$

How do you express their joint PDF?

$$ f_{\mathbf xy}(\mathbf {x,y})\, = \, ? $$

  • 1
    $\begingroup$ Assuming independence? $\endgroup$ – Sasha Mar 17 '13 at 13:31
  • 1
    $\begingroup$ If $X$ and $Y$ are Gaussian variables, then you don't know that $(X,Y)$ is a two-dimensional Gaussian vector. $\endgroup$ – Stefan Hansen Mar 17 '13 at 13:31
  • 1
    $\begingroup$ Relevant example from wikipedia, explaining Stefan's comment. $\endgroup$ – Sasha Mar 17 '13 at 13:32
  • 1
    $\begingroup$ @hkBattousai: Then your initial example is wrong. Just by stating $f_X$ and $f_Y$ as Gaussian densities, you do not know that their joint PDF is of the form you write. $\endgroup$ – Stefan Hansen Mar 17 '13 at 13:34
  • 1
    $\begingroup$ I think one should not use the same notation to refer both to the random variable itself and the argument to the density function. I would often use $\mathbf{X}$ for the random variable and $\mathbf{x}$ for the argument to the density or to a cdf. $\endgroup$ – Michael Hardy Mar 17 '13 at 14:31

In the case where you only assume that $\mathbf{X}$ and $\mathbf{Y}$ are marginally Gaussian, you can't say much about the joint density of $(\mathbf{X},\mathbf{Y})$, and you certainly can't conclude that the joint density is a Gaussian density. In the answer below I've added the additional assumption that the joint distribution is indeed Gaussian.

Assume that $\mathbf{X}=(X_1,\ldots,X_k)\sim\mathcal{N}_k(\boldsymbol{\mu}_x,\mathbf{\Sigma}_x)$ and $\mathbf{Y}=(Y_1,\ldots,Y_k)\sim\mathcal{N}_k(\boldsymbol{\mu}_y,\mathbf{\Sigma}_y)$ and that $\mathbf{Z}=(\mathbf{X},\mathbf{Y})$ follows a Gaussian distribution, then the joint density is given by $$ f_\mathbf{Z}(\mathbf{z})= \frac{1}{(2\pi)^{2k/2}|\boldsymbol\Sigma_z|^{1/2}} \exp\left(-\frac{1}{2}({\mathbf z}-{\boldsymbol\mu_z})^T{\boldsymbol\Sigma_z}^{-1}({\mathbf z}-{\boldsymbol\mu_z}) \right),\quad\mathbf{z}\in\mathbb{R}^{2k}, $$ where $$ \boldsymbol{\mu_z}=(\boldsymbol{\mu}_x,\boldsymbol{\mu}_y) $$ and $$ \mathbf{\Sigma}_z=\begin{bmatrix} \mathbf{\Sigma}_x & \mathbf{\Sigma}_{xy} \\ \mathbf{\Sigma}_{yx} & \mathbf{\Sigma}_y \end{bmatrix} $$ written in block form. Here $\mathbf{\Sigma}_{xy}$ is the $k\times k$ matrix whose $(i,j)$th entry is $$ \mathbf{\Sigma}_{xy}^{i,j}=\mathrm{Cov}(X_i,Y_j). $$

Note that this also holds when $\mathbf{X}$ is $k$-dimensional and $\mathbf{Y}$ is $n$-dimensional for $n\neq k$.

| cite | improve this answer | |
  • $\begingroup$ What does $\boldsymbol{\mu_z}=(\boldsymbol{\mu}_x,\boldsymbol{\mu}_y)$ mean? Did you mean $\mathbf{\mu_z} = [\mathbf{\mu_x}^T \; \mathbf{\mu_y}^T]^T$? $\endgroup$ – hkBattousai Mar 17 '13 at 13:53
  • $\begingroup$ If $\boldsymbol{\mu}_x$ and $\boldsymbol{\mu}_y$ are column vectors, then yes. $\endgroup$ – Stefan Hansen Mar 17 '13 at 13:58
  • $\begingroup$ I'd add that the answer above is correct ONLY if you assume joint normality of $(x,y)$. Nothing about the joint, as opposed to marginal, distributions was mentioned in the question. That's a gap that should be mentioned in any answer. $\endgroup$ – Michael Hardy Mar 17 '13 at 14:29
  • 1
    $\begingroup$ @MichaelHardy: I did actually mention this in my answer (in fact it's emphasized by a bold "and"). Also, in the comments to the question I already mentioned that you cannot say anything in general about the joint distribution of $(X,Y)$ given just the marginals. $\endgroup$ – Stefan Hansen Mar 17 '13 at 15:16
  • $\begingroup$ @StefanHansen : But you didn't say explicitly that the question doesn't mention joint normality. I'd have mentioned that. $\endgroup$ – Michael Hardy Mar 17 '13 at 15:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.