1
$\begingroup$

I got this from the wikipedia page on uncorrelatedness. How does one go from the penultimate term to the last one, i.e.

$$K_{\bf XX}=\text{cov}[\bf X,X] =E[(\mathbf{X}-E[\mathbf{X}])(\mathbf{X}-E[\mathbf{X}])^T]= E[\mathbf{XX}^T]-E[\mathbf{X}]E[\bf X]^T$$

$\endgroup$

2 Answers 2

2
$\begingroup$

$$E[(\mathbf{X}-E[\mathbf{X}])(\mathbf{X}-E[\mathbf{X}])^T]\\ =E[(\mathbf{X}-E[\mathbf{X}])(\mathbf{X}^T-E[\mathbf{X}]^T)]\\ =E[\mathbf{XX}^T-E[\mathbf{X}]\mathbf{X}^T - \mathbf{X}E[\mathbf X]^T+E[\mathbf{X}]E[\mathbf X]^T]\qquad (1)$$ At this point, to avoid cluster of notation, denote $\mu=E[\mathbf X]\implies \mu^T=E[\mathbf X]^T=E[\mathbf X^T]$
which is a constant ($\underline{\text{expectation of a constant is the constant itself}}$), so that using the linearity of expectation, $$\text{terms with $\mu$ inside $E[...]$ should simplify as }\ E[ E[\mathbf X] \mathbf X^T]=E[\mu \mathbf X^T]=\mu E[\mathbf X^T]$$ so that $(1)$ simplifies as $$\begin{aligned}&E[\mathbf{XX}^T-E[\mathbf{X}]\mathbf{X}^T - \mathbf{X}E[\mathbf X]^T+E[\mathbf{X}]E[\mathbf X]^T]\\ =&E[\mathbf{XX}^T-\mu\mathbf{X}^T - \mathbf{X}\mu^T+\mu\mu^T]\\ =&E[\mathbf{XX}^T]-\mu E[\mathbf X^T]-E[\mathbf X]\mu^T+E[\mu\mu^T]\\ =&E[\mathbf{XX}^T]-\mu\mu^T-\mu\mu^T+\mu\mu^T \ (\text{from underlined fact, } E[\mu\mu^T]=\mu\mu^T)\\ =&E[\mathbf{XX}^T]-2\mu\mu^T+\mu\mu^T\\ =&E[\mathbf{XX}^T]-\mu\mu^T\\ =&E[\mathbf{XX}^T]-E[\mathbf{X}]E[\mathbf{X}]^T\end{aligned}$$

$\endgroup$
1
$\begingroup$

Expanding the middle term:

$E[(\mathbf{X}-E[\mathbf{X}])(\mathbf{X}-E[\mathbf{X}])^T] = E[(\mathbf{X}\mathbf{X}^T -\mathbf{X}E(\mathbf{X})^T - E(\mathbf{X})\mathbf{X}^T + E(\mathbf{X})E(\mathbf{X})^T]$

The operator $E$ is linear, so this becomes

$E[\mathbf{X}\mathbf{X}^T] -E[\mathbf{X}]E[\mathbf{X}]^T -E[\mathbf{X}]E[\mathbf{X}]^T +E[\mathbf{X}]E[\mathbf{X}]^T = E[\mathbf{X}\mathbf{X}^T] -E[\mathbf{X}]E[\mathbf{X}]^T$

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.