Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Given a random vector $\mathbf x \sim N(\mathbf{\bar x}, \mathbf{C_x})$ with normal distribution. $\mathbf{\bar x}$ is the mean value vector and $\mathbf{C_x}$ is the covariance matrix of $\mathbf{x}$.

An affine transformation is applied to the $\mathbf{x}$ vector to create a new random $\mathbf{y}$ vector:

$$ \mathbf{y} = \mathbf{Ax} + \mathbf{b} $$

Can we find mean value $\mathbf{\bar y}$ and covariance matrix $\mathbf{C_y}$ of this new vector $\mathbf{y}$ in terms of already given parameters ($\mathbf{\bar x}$, $\mathbf{C_x}$, $\mathbf{A}$ and $\mathbf{b}$)?

Can you please show the steps. Once I learn the method, I will use it on several other distributions myself.

share|cite|improve this question
Hint: $\bar{\mathbf{y}} = E[\mathbf y] = E[\mathbf{Ax}+\mathbf{b}]$. Now apply linearity of expectation. $\mathbf{C_y} = E[(\mathbf{y}-\bar{\mathbf{y}})(\mathbf{y}-\bar{\mathbf{y}})^T] = E[\mathbf{y}\mathbf{y}^T]-E[\bar{\mathbf{y}}\bar{\mathbf{y}}^T]$. – Dilip Sarwate Mar 17 '13 at 2:46

We find the mean of $\mathbf{y}$ by using the fact that $E\{\}$ is a linear operator.

$$ \mathbf{\bar{y}} = E\{\mathbf{y}\} = E\{\mathbf{A}\mathbf{x}+\mathbf{b}\} = \mathbf{A}E\{\mathbf{x}\}+\mathbf{b} = \mathbf{A}\mathbf{\bar{x}}+\mathbf{b} $$

Then we find covariance of

$$ \begin{array}{rcl} \mathbf{C_y} & \triangleq & E\{(\mathbf{y}-\mathbf{\bar{y}})(\mathbf{y}-\mathbf{\bar{y}})^T\} \\ & = & E \Big\{ \Big[ (\mathbf{A}\mathbf{x}+\mathbf{b})-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \Big] \Big[ (\mathbf{A}\mathbf{x}+\mathbf{b})-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \Big] ^T \Big\} \\ & = & E \Big\{ \Big[ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) \Big] \Big[ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) \Big] ^T \Big\} \\ & = & E \Big\{ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) (\mathbf{x}-\mathbf{\bar{x}})^T \mathbf{A}^T \Big\} \\ & = & \mathbf{A} E \Big\{ (\mathbf{x}-\mathbf{\bar{x}}) (\mathbf{x}-\mathbf{\bar{x}})^T \Big\} \mathbf{A}^T \\ & = & \mathbf{A}\mathbf{C_x}\mathbf{A}^T \end{array} $$

Then, $\mathbf{y}$ is defined as,

$$ \mathbf{y} \sim N(\mathbf{A}\mathbf{\bar{x}+\mathbf{b}, \mathbf{A}\mathbf{C_x}\mathbf{A}^T}) $$

That is,

$$ f_\mathbf{Y}(\mathbf{y)} = {1 \over (2\pi)^{N \over 2} |\mathbf{A}\mathbf{C_x}\mathbf{A}^T|^{1 \over 2}} e^{- {1 \over 2} \big[\mathbf{y}-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \big]^T (\mathbf{A}\mathbf{C_x}\mathbf{A}^T)^{-1} \big[\mathbf{y}-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \big] } $$

share|cite|improve this answer
Are there restrictions on A? I'm thinking, for example, about A={{1,1},{0,0}} (Ones are top row, 0s are bottom row) and sigma={{1,0},{0,1}}. Then rank A.sigma.Transpose(A) is 1 and not invertible any more. – BenB Jun 1 '15 at 2:20
@BenB If the rank of $A$ is not full, then the transformation is not linear. And that is not the case mentioned in the question statement. – hkBattousai Feb 6 at 13:24
Not all linear transformations have full rank. If the rank isn't full, the formula $y \sim N(A\bar{x} + b, A C_x A^T)$ still works. The pdf also works if interpreted to use pseudoinverses and pseudodeterminants. – Geoffrey Irving May 8 at 4:58

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.