Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Given a random vector $\mathbf x \sim N(\mathbf{\bar x}, \mathbf{C_x})$ with normal distribution. $\mathbf{\bar x}$ is the mean value vector and $\mathbf{C_x}$ is the covariance matrix of $\mathbf{x}$.

Linear transformation is applied to the $\mathbf{x}$ vector to create a new random $\mathbf{y}$ vector:

$$ \mathbf{y} = \mathbf{Ax} + \mathbf{b} $$

Can we find mean value $\mathbf{\bar y}$ and covariance matrix $\mathbf{C_y}$ of this new vector $\mathbf{y}$ in terms of already given parameters ($\mathbf{\bar x}$, $\mathbf{C_x}$, $\mathbf{A}$ and $\mathbf{b}$)?

Can you please show the steps. Once I learn the method, I will use it on several other distributions myself.

share|cite|improve this question
Hint: $\bar{\mathbf{y}} = E[\mathbf y] = E[\mathbf{Ax}+\mathbf{b}]$. Now apply linearity of expectation. $\mathbf{C_y} = E[(\mathbf{y}-\bar{\mathbf{y}})(\mathbf{y}-\bar{\mathbf{y}})^T] = E[\mathbf{y}\mathbf{y}^T]-E[\bar{\mathbf{y}}\bar{\mathbf{y}}^T]$. – Dilip Sarwate Mar 17 '13 at 2:46

We find the mean of $\mathbf{y}$ by using the fact that $E\{\}$ is a linear operator.

$$ \mathbf{\bar{y}} = E\{\mathbf{y}\} = E\{\mathbf{A}\mathbf{x}+\mathbf{b}\} = \mathbf{A}E\{\mathbf{x}\}+\mathbf{b} = \mathbf{A}\mathbf{\bar{x}}+\mathbf{b} $$

Then we find covariance of

$$ \begin{array}{rcl} \mathbf{C_y} & \triangleq & E\{(\mathbf{y}-\mathbf{\bar{y}})(\mathbf{y}-\mathbf{\bar{y}})^T\} \\ & = & E \Big\{ \Big[ (\mathbf{A}\mathbf{x}+\mathbf{b})-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \Big] \Big[ (\mathbf{A}\mathbf{x}+\mathbf{b})-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \Big] ^T \Big\} \\ & = & E \Big\{ \Big[ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) \Big] \Big[ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) \Big] ^T \Big\} \\ & = & E \Big\{ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) (\mathbf{x}-\mathbf{\bar{x}})^T \mathbf{A}^T \Big\} \\ & = & \mathbf{A} E \Big\{ (\mathbf{x}-\mathbf{\bar{x}}) (\mathbf{x}-\mathbf{\bar{x}})^T \Big\} \mathbf{A}^T \\ & = & \mathbf{A}\mathbf{C_x}\mathbf{A}^T \end{array} $$

Then, $\mathbf{y}$ is defined as,

$$ \mathbf{y} \sim N(\mathbf{A}\mathbf{\bar{x}+\mathbf{b}, \mathbf{A}\mathbf{C_x}\mathbf{A}^T}) $$

That is,

$$ f_\mathbf{Y}(\mathbf{y)} = {1 \over (2\pi)^{N \over 2} |\mathbf{A}\mathbf{C_x}\mathbf{A}^T|^{1 \over 2}} e^{- {1 \over 2} \big[\mathbf{y}-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \big]^T (\mathbf{A}\mathbf{C_x}\mathbf{A}^T)^{-1} \big[\mathbf{y}-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \big] } $$

share|cite|improve this answer
Are there restrictions on A? I'm thinking, for example, about A={{1,1},{0,0}} (Ones are top row, 0s are bottom row) and sigma={{1,0},{0,1}}. Then rank A.sigma.Transpose(A) is 1 and not invertible any more. – BenB Jun 1 '15 at 2:20

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.