# sample mean, without replacement

Going through a stats course and having some difficulties with point estimation. It makes sense in basic examples but when introducing "without replacement" I'm getting stuck.

A box has 4 numbered balls, 1,2,3 and 4. A sample of 2 is drawn without replacement. What is the expected sample mean?

For 1 ball its $\frac{1}{4} \sum X_i$ which is $2.5$. Simple.

Once one ball is extracted, the probabilities change for the others, but as I don't know which ball number was removed, I can't use the same formula.

Would really appreciate some advice and pointers. Thanks.

There are $\binom{4}{2} = 6$ ways to pick 2 balls from 4. Each pair thus has a $\frac{1}{6}$ probability of being picked. The possible averages are $\frac{1+2}{2}, \frac{1+3}{2},\frac{1+4}{2}, \frac{2+3}{2},\frac{2+4}{2},\frac{3+4}{2}$. Denoting the first ball by $X_1$ and the second by $X_2$, by the definition of expectation, \begin{align*} E[\frac{X_1+X_2}{2}] = \frac{1}{6} (3/2 + 4/2 + 5/2 + 5/2 + 6/2 + 7/2) \end{align*}
The expected sample mean will always be the actual mean (here, $2.5$). This is because the expected value of the first ball is $2.5$ and the expected value of the second ball (if you don't know what the first ball was) is also $2.5$. It is always true that $E[X_1+X_2]=E[X_1]+E[X_2]$, even when $X_1$ and $X_2$ are not independent, and so $E[\frac12(X_1+X_2)]=\frac12(E[X_1]+E[X_2])$. This works for any sample size.
If $x_1, x_2, \dots x_N$ is the population and $X_1, X_2, \dots X_n$ is a sample without replacement where $n \leq N$, then $$E\left[ \frac{1}{n} \sum^n_{i=1} X_i \right] = \mu$$ where $\mu =\frac{1}{N} \sum^N_{i=1}x_i$. Proof:
Let $1_{\lbrace x_i \in S \rbrace}$ be a random indicator variable indicating whether $x_i$ is in the sample $S$ of size $n$. We have that $$\sum_{i=1}^{n} X_i = \sum_{i = 1}^N x_i 1_{\lbrace x_i \in S \rbrace}$$ Since $n$ and $N$ are fixed and every sample is equally likely, we know that $$E[1_{\lbrace x_i \in S \rbrace}] = \frac{n}{N}.$$ Therefore, $$E \left[\frac{1}{n} \sum_{i=1}^{n} X_i \right] = E \left[\frac{1}{n} \sum_{i = 1}^N x_i 1_{\lbrace x_i \in S \rbrace} \right] = \frac{1}{n} \sum_{i = 1}^N x_i E[1_{\lbrace x_i \in S \rbrace}] = \mu$$