# Got stuck proving that covariance matrices are positive definite

I got stuck trying to understand the proof of the following statement:

Statement: The covariance matrix $C=\text{Cov}_X$ of a random vector $X$ is positive definite, that is $$\sum_{i,j}^dC_{ij}z_iz_j \geq 0, \ \text{for all } (z_1,z_2, \dots, z_d) \in \mathbb{R}^d.$$ Proof: Set $Y=\sum_{i=1}^d \{ z_i (X_i-EX_i) \}$ for $(z_1, \dots, z_d) \in \mathbb{R}^d$. Then $$0 \leq EY^2 = E\sum_{i,j} \{ z_i z_j (X_i-EX_i ) (X_j -EX_j) \}$$ $$=\sum_{i,j} \{ z_i z_j E \{ (X_i-EX_i)(X_j-EX_j) \}\} = \sum_{i,j} C_{ij}z_iz_j,$$ where by definition of the covariance matrix $$C_{ij}=E \{ (X_i-EX_i)(X_j-EX_j)\}.$$

So, the part I don't understand is the following equality $\rightarrow$ $$EY^2 = E\sum_{i,j} \{ z_i z_j (X_i-EX_i ) (X_j -EX_j) \}.$$

Because, when I try to work on the LFH of the previous equality, I get: $$\begin{split} EY^2 & =E \left( \sum_{i=1}^d \{ z_i (X_i-EX_i) \} \right)^2 \\ & = E \left( \sum_{i=1}^d \{ z_i^2 (X_i-EX_i)^2 \} + \sum_{i,j=1}^d \{ z_i z_j (X_i-EX_i ) (X_j -EX_j) \} \right) \\ & = \sum_{i=1}^d z_i^2 E(X_i-EX_i)^2 + E\sum_{i,j=1}^d \{ z_i z_j (X_i-EX_i ) (X_j -EX_j) \} \\ & \geq E\sum_{i,j=1}^d \{ z_i z_j (X_i-EX_i ) (X_j -EX_j) \} \\ \end{split}$$

What am I doing wrong? Thanks for any help!

In the second line of your work, you split the sum into $i=j$ and $i\neq j$ pieces; this is often a useful technique. But then you treat the $i\neq j$ part as $$\sum_{i,j=1}^d \{ z_i z_j (X_i-EX_i ) (X_j -EX_j) \}$$ instead of $$\sum_{i\neq j} \{ z_i z_j (X_i-EX_i ) (X_j -EX_j) \}$$
The difference of the two expressions is precisely what you are off by in the third line, namely, you have double counted $$\sum_{i=1}^d z_i^2 E(X_i-EX_i)^2$$