Is there some basic examples to show that there are difference meaning between X and E(X) in the covariance formula?
Suppose $X$ and $Y$ are random variables which can take the following pairs of values with the following probabilities
Then $E[X]$, the expected value of $X$, is $0.6 \times 10 + 0.3 \times 30 +0.1 \times 40 = 19$.
Similarly $E[Y]$, the expected value of $X$, is $0.6 \times 20 + 0.3 \times 50 +0.1 \times 10 = 28$.
So the covariance $E\left[(X-E[X])(Y-E[Y])\right]$, the expected value of the products of the differences between the random variables and their expected values, is $$0.6 \times (10-19)\times (20-28) + 0.3 \times (30-19)\times ( 50-28)+0.1 \times (40 -19) \times (10-28) = 78.$$
The value $X$ represents a randomly-varying quantity. The value $E(X)$, if known, represents its average. For example, toss a fair coin 100 times and let $X$ denote the number of heads you see. In this case, if the coin is fair, you would reasonably expect $E(X) = 50$. The value $X$ will vary around 50 as you repeat the experiment.