1
$\begingroup$

Given is a sample $S = {X_1,...,X_n}$ from a target population $P$.

The sampling is done by Simple Random Sampling (random sampling without replacement); n is the sample size.

The sample mean is defined as $\bar{X} = \frac{1}{n}\sum_{i=1}^n X_i$

The following two theorems are given:

(1) $E[\bar{X_n}] = \mu$, with $\mu$ the population mean; that is, the estimator of the mean is unbiased

(2) $V[\bar{X_n}]$ = $\frac{1}{n}\sum_{i=1}^n\sum_{j=1}^n Cov(X_i, X_j)$

My question:

When we calculate the variance of $\bar{X_n}$ we assume that the $X_i$ are dependent. This is how I explain it to myself: If we have a population of different-colored balls, after drawing a blue ball and not putting it back, the probability of drawing a next ball with some color has changed.

How is this dependence not reflected when we calculate $E[\bar{X_n}]$, where it is assumed that all $X_i$ have the same expectation $E[X_i]=\mu$? That is, how do the expectations not change when the sampling is done without replacement?

$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

The only assumption here is that the $X_i$ are identically distributed; i.e., they are drawn from the same population.

It is important to remember that linearity of expectation does not require independence of the variables; e.g., $$\operatorname{E}[A+B] = \operatorname{E}[A]+\operatorname{E}[B]$$ even if $A$ and $B$ are jointly dependent random variables, provided that their individual expectations exist.

Therefore, the theorems follow even when the $X_i$ are dependent but identically distributed. When they are independent, then the covariance is $$\operatorname{Cov}[X_i X_j] = \begin{cases} \operatorname{Var}[X_i], & i = j \\ 0, & i \ne j \end{cases}$$ but this is not assumed.

$\endgroup$
2
  • $\begingroup$ Thanks! I knew that the linearity of expectation does not require independence. So, when we do have sampling without replacement, the samples are indeed dependent, is that correct? And even with the linearity of E[X] where we arrive at $\frac{1}{n} \sum_i E[X_i]$, are, say E[X_1] and E[X_2] not different since the probabilities for the values of X_2 change as we have drawn X_1, or what am I conceptually getting wrong? $\endgroup$
    – TestGuest
    May 22, 2020 at 2:12
  • 1
    $\begingroup$ The samples are dependent. But, because their distributions are identical, therefore their expectations are identical too. $${}$$ Although, the conditional probability function for $X_2$ when given knowledge about $X_1$ will be different from the probability function for $X_1$, notice that we are evaluating the unconditional expectation, not that under any given conditions, and the variables have identical marginal probability functions. $\endgroup$
    – Graham Kemp
    May 22, 2020 at 4:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .