# Affine transformation applied to a multivariate Gaussian random variable - what is the mean vector and covariance matrix of the new variable?

Given a random vector $\mathbf x \sim N(\mathbf{\bar x}, \mathbf{C_x})$ with normal distribution. $\mathbf{\bar x}$ is the mean value vector and $\mathbf{C_x}$ is the covariance matrix of $\mathbf{x}$.

An affine transformation is applied to the $\mathbf{x}$ vector to create a new random $\mathbf{y}$ vector:

$$\mathbf{y} = \mathbf{Ax} + \mathbf{b}$$

Can we find mean value $\mathbf{\bar y}$ and covariance matrix $\mathbf{C_y}$ of this new vector $\mathbf{y}$ in terms of already given parameters ($\mathbf{\bar x}$, $\mathbf{C_x}$, $\mathbf{A}$ and $\mathbf{b}$)?

Can you please show the steps. Once I learn the method, I will use it on several other distributions myself.

-
Hint: $\bar{\mathbf{y}} = E[\mathbf y] = E[\mathbf{Ax}+\mathbf{b}]$. Now apply linearity of expectation. $\mathbf{C_y} = E[(\mathbf{y}-\bar{\mathbf{y}})(\mathbf{y}-\bar{\mathbf{y}})^T] = E[\mathbf{y}\mathbf{y}^T]-E[\bar{\mathbf{y}}\bar{\mathbf{y}}^T]$. – Dilip Sarwate Mar 17 '13 at 2:46

We find the mean of $\mathbf{y}$ by using the fact that $E\{\}$ is a linear operator.

$$\mathbf{\bar{y}} = E\{\mathbf{y}\} = E\{\mathbf{A}\mathbf{x}+\mathbf{b}\} = \mathbf{A}E\{\mathbf{x}\}+\mathbf{b} = \mathbf{A}\mathbf{\bar{x}}+\mathbf{b}$$

Then we find covariance of

$$\begin{array}{rcl} \mathbf{C_y} & \triangleq & E\{(\mathbf{y}-\mathbf{\bar{y}})(\mathbf{y}-\mathbf{\bar{y}})^T\} \\ & = & E \Big\{ \Big[ (\mathbf{A}\mathbf{x}+\mathbf{b})-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \Big] \Big[ (\mathbf{A}\mathbf{x}+\mathbf{b})-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \Big] ^T \Big\} \\ & = & E \Big\{ \Big[ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) \Big] \Big[ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) \Big] ^T \Big\} \\ & = & E \Big\{ \mathbf{A}(\mathbf{x}-\mathbf{\bar{x}}) (\mathbf{x}-\mathbf{\bar{x}})^T \mathbf{A}^T \Big\} \\ & = & \mathbf{A} E \Big\{ (\mathbf{x}-\mathbf{\bar{x}}) (\mathbf{x}-\mathbf{\bar{x}})^T \Big\} \mathbf{A}^T \\ & = & \mathbf{A}\mathbf{C_x}\mathbf{A}^T \end{array}$$

Then, $\mathbf{y}$ is defined as,

$$\mathbf{y} \sim N(\mathbf{A}\mathbf{\bar{x}+\mathbf{b}, \mathbf{A}\mathbf{C_x}\mathbf{A}^T})$$

That is,

$$f_\mathbf{Y}(\mathbf{y)} = {1 \over (2\pi)^{N \over 2} |\mathbf{A}\mathbf{C_x}\mathbf{A}^T|^{1 \over 2}} e^{- {1 \over 2} \big[\mathbf{y}-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \big]^T (\mathbf{A}\mathbf{C_x}\mathbf{A}^T)^{-1} \big[\mathbf{y}-(\mathbf{A}\mathbf{\bar{x}}+\mathbf{b}) \big] }$$

-
Are there restrictions on A? I'm thinking, for example, about A={{1,1},{0,0}} (Ones are top row, 0s are bottom row) and sigma={{1,0},{0,1}}. Then rank A.sigma.Transpose(A) is 1 and not invertible any more. – BenB Jun 1 '15 at 2:20
@BenB If the rank of $A$ is not full, then the transformation is not linear. And that is not the case mentioned in the question statement. – hkBattousai Feb 6 at 13:24
Not all linear transformations have full rank. If the rank isn't full, the formula $y \sim N(A\bar{x} + b, A C_x A^T)$ still works. The pdf also works if interpreted to use pseudoinverses and pseudodeterminants. – Geoffrey Irving May 8 at 4:58