I got stuck trying to understand the proof of the following statement:

Statement: The covariance matrix $C=\text{Cov}_X$ of a random vector $X$ is positive definite, that is $$\sum_{i,j}^dC_{ij}z_iz_j \geq 0, \ \text{for all } (z_1,z_2, \dots, z_d) \in \mathbb{R}^d.$$ Proof: Set $Y=\sum_{i=1}^d \{ z_i (X_i-EX_i) \} $ for $(z_1, \dots, z_d) \in \mathbb{R}^d$. Then $$0 \leq EY^2 = E\sum_{i,j} \{ z_i z_j (X_i-EX_i ) (X_j -EX_j) \}$$ $$=\sum_{i,j} \{ z_i z_j E \{ (X_i-EX_i)(X_j-EX_j) \}\} = \sum_{i,j} C_{ij}z_iz_j,$$ where by definition of the covariance matrix $$C_{ij}=E \{ (X_i-EX_i)(X_j-EX_j)\}.$$

So, the part I don't understand is the following equality $\rightarrow$ $$EY^2 = E\sum_{i,j} \{ z_i z_j (X_i-EX_i ) (X_j -EX_j) \}.$$

Because, when I try to work on the LFH of the previous equality, I get: \begin{equation} \begin{split} EY^2 & =E \left( \sum_{i=1}^d \{ z_i (X_i-EX_i) \} \right)^2 \\ & = E \left( \sum_{i=1}^d \{ z_i^2 (X_i-EX_i)^2 \} + \sum_{i,j=1}^d \{ z_i z_j (X_i-EX_i ) (X_j -EX_j) \} \right) \\ & = \sum_{i=1}^d z_i^2 E(X_i-EX_i)^2 + E\sum_{i,j=1}^d \{ z_i z_j (X_i-EX_i ) (X_j -EX_j) \} \\ & \geq E\sum_{i,j=1}^d \{ z_i z_j (X_i-EX_i ) (X_j -EX_j) \} \\ \end{split} \end{equation}

What am I doing wrong? Thanks for any help!


In the second line of your work, you split the sum into $i=j$ and $i\neq j$ pieces; this is often a useful technique. But then you treat the $i\neq j$ part as $$\sum_{i,j=1}^d \{ z_i z_j (X_i-EX_i ) (X_j -EX_j) \} $$ instead of $$\sum_{i\neq j} \{ z_i z_j (X_i-EX_i ) (X_j -EX_j) \} $$

The difference of the two expressions is precisely what you are off by in the third line, namely, you have double counted $$ \sum_{i=1}^d z_i^2 E(X_i-EX_i)^2 $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.