Question: Suppose we have: \begin{align*} \begin{bmatrix} X_1 \\ X_2 \end{bmatrix} \sim N\left(\begin{bmatrix} 6 \\ 3 \end{bmatrix}, \begin{bmatrix} 12 & 3 \\ 3 & 2 \end{bmatrix} \right) \end{align*} and let $Y = X_1 - 3X_2$, then what is the joint distribution of $X_1$ and $Y$?

My working: The marginal distribution of $X_1$ is: \begin{align*} X_1 \sim N(6, 12) \end{align*}

The marginal distribution of Y is: $$Y \sim N\left(-3, 12 \right)$$ since \begin{gather*} E\left(Y\right) = 6 - (3)(3) = -3 \\ V\left( Y\right) = 12+ (-3)(2)(-3) + 2(3)(-3) = 12 \end{gather*}

So we have $X_1 \sim N\left(6, 12\right)$ and $Y \sim N\left(-3, 12\right)$, first we need to work out the covariance between $X_1$ and $Y$: \begin{align*} cov\left(X_1,Y\right) & = E(X_1Y) - E(X_1)E(Y) \\ & = E\left(X_1\left(X_1-3X_2 \right) \right) - 6(-3) \\ & = E(X_1^2) - 3E(X_1X_2) + 18 \\ & = V(X_1) + E(X_1)^2 -3\left[cov(X_1, X_2) + E(X_1)E(X_2) \right] + 18 \\ & = 12+36-3\left[3+(6)(3) \right] + 18\\ & = 3 \end{align*} Hence, the joint distribution between $X_1$ and $Y$ is given by: \begin{align*} \begin{bmatrix} X_1 \\ Y \end{bmatrix} \sim N\left(\begin{bmatrix} 6 \\ -3 \end{bmatrix}, \begin{bmatrix} 12 & 3 \\ 3 & 12 \end{bmatrix} \right) \end{align*}


I feel my answer is not right because I assumed that if we have two marginal normal random variables, say, $X_1 \sim N(\mu_1, \Sigma_{11})$ and $X_2 \sim N(\mu_2, \Sigma_{22})$, then $X_1$ and $X_2$ is jointly normal. That is,

\begin{align*} \begin{bmatrix} X_1 \\ X_2 \end{bmatrix} \sim N\left(\begin{bmatrix} \mu_1 \\ \mu_2 \end{bmatrix}, \begin{bmatrix} \Sigma_{11} & cov(X_1, X_2) \\ cov(X_1, X_2) & \Sigma_{22} \end{bmatrix} \right) \end{align*} where $cov(X_1, X_2)$ denotes the covariance between $X_1$ and $X_2$.

However, I know in general just because $X_1$ and $X_2$ are both marginally normal, this does not mean they are jointly normal. How else can I find the joint distribution between $X_1$ and $Y$?



Recall the following fundamental fact about normal distributions:

If a random vector $X$ is normal $(M,\Sigma)$ then, for every matrix $A$, $AX$ is normal $(AM,A\Sigma A^T)$.

In your case, $A=\begin{pmatrix}1 & 0\\ 1 & -3\end{pmatrix}$ and $X=\begin{pmatrix}X_1\\ X_2\end{pmatrix}$ yield $AX=\begin{pmatrix}X_1\\ Y\end{pmatrix}$ hence you simply must compute the mean vector $AM$ and the covariance matrix $A\Sigma A^T$.

Note that the hypothesis in this exercise is not that $X_1$ and $X_2$ are normal but that the random vector $X=\begin{pmatrix}X_1\\ X_2\end{pmatrix}$ is normal. In the former situation, you are right to stress that chaos could ensue, but not in the latter.

  • $\begingroup$ @Thanks Did, that cleared things up! $\endgroup$ – Trts Oct 18 '13 at 7:11
  • $\begingroup$ Great. ${}$ $ $ $\endgroup$ – Did Oct 18 '13 at 7:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.