Independence of Random Variables for Expectation and Variance of Mean Estimator

Question

Given is a sample $S = {X_1,...,X_n}$ from a target population $P$.

The sampling is done by Simple Random Sampling (random sampling without replacement); n is the sample size.

The sample mean is defined as $\bar{X} = \frac{1}{n}\sum_{i=1}^n X_i$

The following two theorems are given:

(1) $E[\bar{X_n}] = \mu$, with $\mu$ the population mean; that is, the estimator of the mean is unbiased

(2) $V[\bar{X_n}]$ = $\frac{1}{n}\sum_{i=1}^n\sum_{j=1}^n Cov(X_i, X_j)$

My question:

When we calculate the variance of $\bar{X_n}$ we assume that the $X_i$ are dependent. This is how I explain it to myself: If we have a population of different-colored balls, after drawing a blue ball and not putting it back, the probability of drawing a next ball with some color has changed.

How is this dependence not reflected when we calculate $E[\bar{X_n}]$, where it is assumed that all $X_i$ have the same expectation $E[X_i]=\mu$? That is, how do the expectations not change when the sampling is done without replacement?

Answer 1

The only assumption here is that the $X_i$ are identically distributed; i.e., they are drawn from the same population.

It is important to remember that linearity of expectation does not require independence of the variables; e.g., $$\operatorname{E}[A+B] = \operatorname{E}[A]+\operatorname{E}[B]$$ even if $A$ and $B$ are jointly dependent random variables, provided that their individual expectations exist.

Therefore, the theorems follow even when the $X_i$ are dependent but identically distributed. When they are independent, then the covariance is $$\operatorname{Cov}[X_i X_j] = \begin{cases} \operatorname{Var}[X_i], & i = j \\ 0, & i \ne j \end{cases}$$ but this is not assumed.