49
$\begingroup$

Given a random vector $\mathbf x \sim N(\mathbf{\bar x}, \mathbf{C_x})$ with normal distribution. $\mathbf{\bar x}$ is the mean value vector and $\mathbf{C_x}$ is the covariance matrix of $\mathbf{x}$.

An affine transformation is applied to the $\mathbf{x}$ vector to create a new random $\mathbf{y}$ vector:

$$ \mathbf{y} = \mathbf{Ax} + \mathbf{b} $$

Can we find mean value $\mathbf{\bar y}$ and covariance matrix $\mathbf{C_y}$ of this new vector $\mathbf{y}$ in terms of already given parameters ($\mathbf{\bar x}$, $\mathbf{C_x}$, $\mathbf{A}$ and $\mathbf{b}$)?

Can you please show the steps. Once I learn the method, I will use it on several other distributions myself.

$\endgroup$
1
  • 2
    $\begingroup$ Hint: $\bar{\mathbf{y}} = E[\mathbf y] = E[\mathbf{Ax}+\mathbf{b}]$. Now apply linearity of expectation. $\mathbf{C_y} = E[(\mathbf{y}-\bar{\mathbf{y}})(\mathbf{y}-\bar{\mathbf{y}})^T] = E[\mathbf{y}\mathbf{y}^T]-E[\bar{\mathbf{y}}\bar{\mathbf{y}}^T]$. $\endgroup$ Commented Mar 17, 2013 at 2:46

1 Answer 1

61
$\begingroup$

We find the mean of $\mathbf{y}$ by using the fact that $\mathbb{E}\{\}$ is a linear operator.

$$ \mathbf{\bar{y}} = \mathbb{E}\{\mathbf{y}\} = \mathbb{E}\{\mathbf{A}\mathbf{x}+\mathbf{b}\} = \mathbf{A}\mathbb{E}\{\mathbf{x}\}+\mathbf{b} = \mathbf{A}\mathbf{\bar{x}}+\mathbf{b} $$

Then we find covariance of

$$ \begin{array}{rcl} \mathbf{C_y} & \triangleq & \mathbb{E}\{(\mathbf{y}-\mathbf{\bar{y}})(\mathbf{y}-\mathbf{\bar{y}})^\top\} \\ & = & \mathbb{E} \Big\{ \Big[ (\mathbf{A}\mathbf{x}+\mathbf{b})-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \Big] \Big[ (\mathbf{A}\mathbf{x}+\mathbf{b})-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \Big] ^\top \Big\} \\ & = & \mathbb{E} \Big\{ \Big[ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) \Big] \Big[ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) \Big] ^\top \Big\} \\ & = & \mathbb{E} \Big\{ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) (\mathbf{x}-\mathbf{\bar{x}})^\top \mathbf{A}^\top \Big\} \\ & = & \mathbf{A} \mathbb{E} \Big\{ (\mathbf{x}-\mathbf{\bar{x}}) (\mathbf{x}-\mathbf{\bar{x}})^\top \Big\} \mathbf{A}^\top \\ & = & \mathbf{A}\mathbf{C_x}\mathbf{A}^\top \end{array} $$

Then, $\mathbf{y}$ is defined as,

$$ \mathbf{y} \sim \mathcal{N}(\mathbf{A}\mathbf{\bar{x}+\mathbf{b}, \mathbf{A}\mathbf{C_x}\mathbf{A}^\top}) $$

That is,

$$ f_\mathbf{Y}(\mathbf{y)} = {1 \over \sqrt{\lvert2\pi\mathbf{A}\mathbf{C_x}\mathbf{A}^\top\rvert}} \exp\left(- {1 \over 2} \big[\mathbf{y}-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \big]^\top (\mathbf{A}\mathbf{C_x}\mathbf{A}^\top)^{-1} \big[\mathbf{y}-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \big] \right) $$

$\endgroup$
4
  • $\begingroup$ Are there restrictions on A? I'm thinking, for example, about A={{1,1},{0,0}} (Ones are top row, 0s are bottom row) and sigma={{1,0},{0,1}}. Then rank A.sigma.Transpose(A) is 1 and not invertible any more. $\endgroup$
    – BenB
    Commented Jun 1, 2015 at 2:20
  • $\begingroup$ @BenB If the rank of $A$ is not full, then the transformation is not linear. And that is not the case mentioned in the question statement. $\endgroup$ Commented Feb 6, 2016 at 13:24
  • 8
    $\begingroup$ Not all linear transformations have full rank. If the rank isn't full, the formula $y \sim N(A\bar{x} + b, A C_x A^T)$ still works. The pdf also works if interpreted to use pseudoinverses and pseudodeterminants. $\endgroup$ Commented May 8, 2016 at 4:58
  • $\begingroup$ To be even more complete this answer could include the cross covariance! nice answer btw $\endgroup$ Commented Feb 26, 2018 at 13:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .