Let $X \text{~}N(\mu_x, \sigma_x^2)$ and $ Y \text{~}N(\mu_y, \sigma_y^2)$ be two correlated normal random variables and $U = \frac{X - \mu_x}{\sigma_x}$ and $U = \frac{Y - \mu_x}{\sigma_x}$ be two normalized random variables with $\text{Cov(U,V)} = \rho = \frac{\sigma_{x,y}^2}{\sigma_x^2\sigma_y^2}$ where $\sigma_{xy}^2$ is the covariance of $X$ and $Y$.

The book I'm reading states that the joint density derived of $U$ and $V$ is $$f_{UV}(u,v) = \frac{1}{2\pi \sqrt{1-\rho^2}}\text{exp}(-\frac{u^2 - 2\rho u v + v^2}{2(1-p^2)})$$

How is this joint density derived? And how can I convert this density from $UV$ to $XY$?


The multivariate normal distribution of a vector $x\in \mathbf{R}^n$ is defined as $$f(x) = \frac{1}{(2\pi)^n |\Sigma|} e^{-1/2 \cdot (x-\mu)^T\Sigma^{-1}(x-\mu)}$$ where $\Sigma\in \mathbf{R}^{n\times n}$ is the covariance matrix where $\Sigma_{ij} = \mathbf{Cov}(x_i,x_j)$ (note the special case when $i=j$ is just $\mathbf{var}(x_i)$), $|\Sigma|$ is the determinant of the matrix, and $\mu\in\mathbf{R}^n$ is the mean.

With all this out of the way, the answer to your second question about how to represent the distribution in $XY$ rather than $UV$ is already answered by the definition of the normal distribution. Specifically, $$\mu = [\mu_x, \mu_y]^T$$ $$\Sigma = \begin{bmatrix}\sigma_x^2 & \sigma_{xy}^2\\ \sigma_{yx}^2 & \sigma_y^2\end{bmatrix}$$ $$|\Sigma| = \sigma_x^2\sigma_y^2 - \sigma_{xy}^2 \cdot \sigma_{yx}^2$$ Plug in $x = [X,Y]^T$ and that is distribution.

For your first question, notice that we can relate $UV$ and $XY$ by a linear (technically affine) transformation $$\begin{bmatrix} U\\ V \end{bmatrix} = \begin{bmatrix} 1/\sigma_x & 0\\ 0 & 1/\sigma_y \end{bmatrix} \begin{bmatrix} X\\ Y \end{bmatrix} + \begin{bmatrix} -\mu_x /\sigma_x\\ -\mu_y / \sigma_y \end{bmatrix} $$ More succinctly, we have $u = Ax + b$. It just so happens that a linear combination (plus a possible constant) of Gaussian random variables, is in fact Gaussian (this is not obvious). And since we know the Gaussian PDF is defined by the two parameters $\Sigma$ and $\mu$, if we can find those, then we are done.

It should be clear from linearity of expectation that the mean of $u\in \mathbf{R}^2$ is just $A\mu +b$. For the covariance we have the identity $\mathbf{cov}(Az + b) = A\mathbf{cov}(z)A^T$ (this also more or less falls out from linearity of expectation and you can find numerous proofs online)

Doing everything out we find that the distribution of $UV$ is Gaussian with mean vector $$\mu_{UV} = \begin{bmatrix} 1/\sigma_x & 0\\ 0 & 1/\sigma_y \end{bmatrix} \begin{bmatrix} \mu_x\\ \mu_y \end{bmatrix} + \begin{bmatrix} -\mu_x /\sigma_x\\ -\mu_y / \sigma_y \end{bmatrix} = \begin{bmatrix} 0\\ 0 \end{bmatrix} $$ $$\Sigma_{UV} = \begin{bmatrix} 1/\sigma_x & 0\\ 0 & 1/\sigma_y \end{bmatrix} \begin{bmatrix}\sigma_x^2 & \sigma_{xy}^2\\ \sigma_{yx}^2 & \sigma_y^2 \end{bmatrix} \begin{bmatrix} 1/\sigma_x & 0\\ 0 & 1/\sigma_y \end{bmatrix} = \begin{bmatrix} 1 & \rho \\ \rho & 1 \end{bmatrix} $$

You can go ahead and compute the inverse and determinant and see that you get the joint PDF you have described

  • $\begingroup$ Can you expand on what you mean by "plug in $x=[X,Y]^T$ and that is the distribution"? By substituting in $(x,y)$ for $(u,v)$ in $f_{UV}(u,v)$ I don't see why that would be the distribution. $\endgroup$ – Oliver G Apr 28 '18 at 13:41
  • $\begingroup$ The distribution is given by $f(x)$ (see my answer above) where $x$ is a column vector $[X,Y]^T$ (i.e. $x$ is not the same $x$ you are using). You know all the parameters so you just substitute in all the values. $f_{UV}(u,v)$ does not come into play $\endgroup$ – Casey Apr 28 '18 at 18:21
  • $\begingroup$ But by substituting in all the values then you're just writing $f_{XY}(x,y)$ and not inverting $f_{UV}(u,v)$ to get $f_{XY}(x,y)$. Given the information in the box of my question how do you invert $f_{UV}(u,v)$ to get $f_{XY}(x,y)$? $\endgroup$ – Oliver G Apr 28 '18 at 20:27
  • $\begingroup$ Exactly, there is no need to transform $f_{UV}(u,v)$ to $f_{XY}(x,y)$. The problem states the properties of the joint density of $X$ and $Y$ and that is the end of the story. Going from $UV$ to $XY$ is circular. In order to get $UV$ you have to know the distribution of $XY$ in the first place. If you insist on doing it, purely as an exercise, then you can apply everything above to $x = A^{-1}u -A^{-1}b$ $\endgroup$ – Casey Apr 29 '18 at 2:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.