Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I was just reading the wikipedia article about Multivariate normal distribution: http://en.wikipedia.org/wiki/Multivariate_normal_distribution

I use a little bit different notation. If $X_1,...,X_n$ are independent $\mathcal{N}(0,1)$ random variables, $X=(X_1,...,X_n)$ and $m=(m_1,...m_n)$ are n-dimensional vectors and $B$ is a $n\times n$ matrix, then $Y=m+BX$ has density function

$$f_Y(x)=\frac{1}{\sqrt{(2\pi)^n|\boldsymbol\Sigma|}} \exp\left(-\frac{1}{2}({x}-{m})^T{\boldsymbol\Sigma}^{-1}({x}-{m}) \right)$$

where $\Sigma=BB^T$

Question: In the wikipedia article its part of the definition that $\Sigma$ is the covariance matric from $Y$, i.e $\sum_{ij}=Cov(Y_i,Y_j)$, but why? How can this be proved.

share|improve this question

1 Answer 1

up vote 2 down vote accepted

By the definition, any covariance matrix $\Sigma$ consists of $\Sigma_{i,j} = Cov(X_i, X_j) = E[(X_i - \mu_i)(X_j - \mu_j)]$, where randoms $X_i$ and $X_j$ have expectations $E[X_i] = \mu_i$ and $E[X_j] = \mu_j$ accordingly. So your problem is to prove $Cov(m_i + \mathbf{b}_i \mathbf{X}, m_j + \mathbf{b}_j \mathbf{X}) = (B \cdot \Sigma \cdot B^\top)_{i,j}$, where $B$ is non-singular matrix with rows $\mathbf{b}_i = (b_{i,1}, \ldots, b_{i,n})$, $1 \le i \le n$ and $\mathbf{X} = (X_1, \ldots, X_n)$ is $n$-dimensional random, $(m_1, \ldots, m_n)$ is vector of scalars.

Let $\mathbf{X}$ has the covariance $n \times n$-matrix $\Sigma$, which is symmetric.

Note, $E[m_i + \mathbf{b}_i \mathbf{X}] = m_i + \mathbf{b}_i \mathbf{\mu}$, where $\mathbf{\mu} = (\mu_1, \ldots, \mu_n)$. Expectation operator $E$ is linear. So, the proof:

$Cov(m_i + \mathbf{b}_i \mathbf{X}, m_j + \mathbf{b}_j \mathbf{X}) = E[(\mathbf{b}_i (\mathbf{X} - \mathbf{\mu}))(\mathbf{b}_j (\mathbf{X} - \mathbf{\mu}))] = E[\sum_{t=1}^n \sum_{p=1}^n b_{i,t} b_{j,p} (X_t - \mu_t)(X_p - \mu_p)] = \sum_{t=1}^n \sum_{p=1}^n b_{i,t} b_{j,p} E[(X_t - \mu_t)(X_p - \mu_p)] = \sum_{t=1}^n b_{i,t} \left(\sum_{p=1}^n b_{j,p} \Sigma_{p,t}\right) = \sum_{t=1}^n b_{i,t} \left( B \cdot \Sigma \right)_{j,t} = \left(B \cdot \Sigma \cdot B^\top\right)_{i,j}$

So the covariance matrix of $\mathbf{m} + B \cdot \mathbf{X}$ is $B \cdot \Sigma \cdot B^\top$, where $Cov(\mathbf{X}) = \Sigma$.

share|improve this answer
1  
Still, I do not see the relation $\sum_{ij}=Cov(Y_i,Y_j)$, I think this can be proved formally somehow. –  Babla May 18 '13 at 15:20
    
@Babla $Cov(\mathbf{c} + B \cdot \mathbf{X}, \mathbf{c} + B \cdot \mathbf{X}) = B \cdot \Sigma \cdot B^\top$, where $\Sigma$ is covariance matrix of random Gaussian vector $\Sigma$. May be I do not understand your problem... –  Piotr Semenov May 18 '13 at 15:26
    
I think this is what we need to show. $Cov(Y_i,Y_j)=Cov(m+BX_i,m+BX_j)=Cov(BX_i,BX_j)=?$ –  Babla May 18 '13 at 15:47
    
@Babla I see. I have rewritten my answer according to your comment. –  Piotr Semenov May 18 '13 at 16:56
    
Thanks you, thats what I was looking for. I have one more question. If I have two ranndom variables $Y_1$ ($\mathcal{N}(\mu_1,\sigma_1^2$)) and $Y_2$ ($\mathcal{N}(\mu_2,\sigma_2^2$) and we also know that $Cov(Y_1,Y_2)$, how can follow from the observations above that $Y_1$ and $Y_2$ are independent? –  Babla May 18 '13 at 17:15

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.