27
$\begingroup$

Given a random vector $\mathbf x \sim N(\mathbf{\bar x}, \mathbf{C_x})$ with normal distribution. $\mathbf{\bar x}$ is the mean value vector and $\mathbf{C_x}$ is the covariance matrix of $\mathbf{x}$.

An affine transformation is applied to the $\mathbf{x}$ vector to create a new random $\mathbf{y}$ vector:

$$ \mathbf{y} = \mathbf{Ax} + \mathbf{b} $$

Can we find mean value $\mathbf{\bar y}$ and covariance matrix $\mathbf{C_y}$ of this new vector $\mathbf{y}$ in terms of already given parameters ($\mathbf{\bar x}$, $\mathbf{C_x}$, $\mathbf{A}$ and $\mathbf{b}$)?

Can you please show the steps. Once I learn the method, I will use it on several other distributions myself.

$\endgroup$
  • 2
    $\begingroup$ Hint: $\bar{\mathbf{y}} = E[\mathbf y] = E[\mathbf{Ax}+\mathbf{b}]$. Now apply linearity of expectation. $\mathbf{C_y} = E[(\mathbf{y}-\bar{\mathbf{y}})(\mathbf{y}-\bar{\mathbf{y}})^T] = E[\mathbf{y}\mathbf{y}^T]-E[\bar{\mathbf{y}}\bar{\mathbf{y}}^T]$. $\endgroup$ – Dilip Sarwate Mar 17 '13 at 2:46
29
$\begingroup$

We find the mean of $\mathbf{y}$ by using the fact that $\mathbb{E}\{\}$ is a linear operator.

$$ \mathbf{\bar{y}} = \mathbb{E}\{\mathbf{y}\} = \mathbb{E}\{\mathbf{A}\mathbf{x}+\mathbf{b}\} = \mathbf{A}\mathbb{E}\{\mathbf{x}\}+\mathbf{b} = \mathbf{A}\mathbf{\bar{x}}+\mathbf{b} $$

Then we find covariance of

$$ \begin{array}{rcl} \mathbf{C_y} & \triangleq & \mathbb{E}\{(\mathbf{y}-\mathbf{\bar{y}})(\mathbf{y}-\mathbf{\bar{y}})^\top\} \\ & = & \mathbb{E} \Big\{ \Big[ (\mathbf{A}\mathbf{x}+\mathbf{b})-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \Big] \Big[ (\mathbf{A}\mathbf{x}+\mathbf{b})-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \Big] ^\top \Big\} \\ & = & \mathbb{E} \Big\{ \Big[ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) \Big] \Big[ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) \Big] ^\top \Big\} \\ & = & \mathbb{E} \Big\{ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) (\mathbf{x}-\mathbf{\bar{x}})^\top \mathbf{A}^\top \Big\} \\ & = & \mathbf{A} \mathbb{E} \Big\{ (\mathbf{x}-\mathbf{\bar{x}}) (\mathbf{x}-\mathbf{\bar{x}})^\top \Big\} \mathbf{A}^\top \\ & = & \mathbf{A}\mathbf{C_x}\mathbf{A}^\top \end{array} $$

Then, $\mathbf{y}$ is defined as,

$$ \mathbf{y} \sim \mathcal{N}(\mathbf{A}\mathbf{\bar{x}+\mathbf{b}, \mathbf{A}\mathbf{C_x}\mathbf{A}^\top}) $$

That is,

$$ f_\mathbf{Y}(\mathbf{y)} = {1 \over \sqrt{\lvert2\pi\mathbf{A}\mathbf{C_x}\mathbf{A}^\top\rvert}} \exp\left(- {1 \over 2} \big[\mathbf{y}-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \big]^\top (\mathbf{A}\mathbf{C_x}\mathbf{A}^\top)^{-1} \big[\mathbf{y}-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \big] \right) $$

$\endgroup$
  • $\begingroup$ Are there restrictions on A? I'm thinking, for example, about A={{1,1},{0,0}} (Ones are top row, 0s are bottom row) and sigma={{1,0},{0,1}}. Then rank A.sigma.Transpose(A) is 1 and not invertible any more. $\endgroup$ – BenB Jun 1 '15 at 2:20
  • $\begingroup$ @BenB If the rank of $A$ is not full, then the transformation is not linear. And that is not the case mentioned in the question statement. $\endgroup$ – hkBattousai Feb 6 '16 at 13:24
  • 4
    $\begingroup$ Not all linear transformations have full rank. If the rank isn't full, the formula $y \sim N(A\bar{x} + b, A C_x A^T)$ still works. The pdf also works if interpreted to use pseudoinverses and pseudodeterminants. $\endgroup$ – Geoffrey Irving May 8 '16 at 4:58
  • $\begingroup$ To be even more complete this answer could include the cross covariance! nice answer btw $\endgroup$ – Marco Aguiar Feb 26 '18 at 13:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.