Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It only takes a minute to sign up.
Sign up to join this community
I got this from the wikipedia page on uncorrelatedness. How does one go from the penultimate term to the last one, i.e.
$$K_{\bf XX}=\text{cov}[\bf X,X] =E[(\mathbf{X}-E[\mathbf{X}])(\mathbf{X}-E[\mathbf{X}])^T]= E[\mathbf{XX}^T]-E[\mathbf{X}]E[\bf X]^T$$
$$E[(\mathbf{X}-E[\mathbf{X}])(\mathbf{X}-E[\mathbf{X}])^T]\\
=E[(\mathbf{X}-E[\mathbf{X}])(\mathbf{X}^T-E[\mathbf{X}]^T)]\\
=E[\mathbf{XX}^T-E[\mathbf{X}]\mathbf{X}^T - \mathbf{X}E[\mathbf X]^T+E[\mathbf{X}]E[\mathbf X]^T]\qquad (1)$$
At this point, to avoid cluster of notation, denote $\mu=E[\mathbf X]\implies \mu^T=E[\mathbf X]^T=E[\mathbf X^T]$
which is a constant ($\underline{\text{expectation of a constant is the constant itself}}$), so that using the linearity of expectation, $$\text{terms with $\mu$ inside $E[...]$ should simplify as }\ E[ E[\mathbf X] \mathbf X^T]=E[\mu \mathbf X^T]=\mu E[\mathbf X^T]$$ so that $(1)$ simplifies as
$$\begin{aligned}&E[\mathbf{XX}^T-E[\mathbf{X}]\mathbf{X}^T - \mathbf{X}E[\mathbf X]^T+E[\mathbf{X}]E[\mathbf X]^T]\\
=&E[\mathbf{XX}^T-\mu\mathbf{X}^T - \mathbf{X}\mu^T+\mu\mu^T]\\
=&E[\mathbf{XX}^T]-\mu E[\mathbf X^T]-E[\mathbf X]\mu^T+E[\mu\mu^T]\\
=&E[\mathbf{XX}^T]-\mu\mu^T-\mu\mu^T+\mu\mu^T \ (\text{from underlined fact, } E[\mu\mu^T]=\mu\mu^T)\\
=&E[\mathbf{XX}^T]-2\mu\mu^T+\mu\mu^T\\
=&E[\mathbf{XX}^T]-\mu\mu^T\\
=&E[\mathbf{XX}^T]-E[\mathbf{X}]E[\mathbf{X}]^T\end{aligned}$$
Expanding the middle term:
$E[(\mathbf{X}-E[\mathbf{X}])(\mathbf{X}-E[\mathbf{X}])^T] = E[(\mathbf{X}\mathbf{X}^T -\mathbf{X}E(\mathbf{X})^T - E(\mathbf{X})\mathbf{X}^T + E(\mathbf{X})E(\mathbf{X})^T]$
The operator $E$ is linear, so this becomes
$E[\mathbf{X}\mathbf{X}^T] -E[\mathbf{X}]E[\mathbf{X}]^T -E[\mathbf{X}]E[\mathbf{X}]^T +E[\mathbf{X}]E[\mathbf{X}]^T = E[\mathbf{X}\mathbf{X}^T] -E[\mathbf{X}]E[\mathbf{X}]^T$
