6
$\begingroup$

In a binomial experiment we know that every trial is is independent and that the probability of success, $p$ is the same in every trial. This also means that the expected value of any individual trial is $p$. So if we have a sample of size $n$, by the linearity property of the expectation, the expected value of the same is just $n \cdot p$. This is all intuitive.

When the population size is finite and when we don't replace the items after every trial, we can't use the binomial distribution to get the probability of $k$ successes in a sample of size $n$ where the population is of size $N$ and the number of successes is $R$ simply because the probability of obtaining a success after every trial changes as the $R$ or/and $N$ change(s). So far so good. Yet when they calculate the expected value of the hypergeometric random variable, it is $(n \cdot R/N)$. This seems to me as the same as saying the probability of obtaining a success in every trial is the same ($R/N$) which is not intuitive at all because I should at least be expecting to see $N$ reducing by $1$ after every trial. I know that there's a flaw in my thinking. Can someone help point it out ?

Edit: I think I'm going to give up on understanding why the expected value of the hypergeometric random variable (HRV) is at it is. None of the answers have alleviated my confusion. I don't think I've made my confusion clear enough. My problem is I'm going about the process of finding the expected value of the HRV in the same way as that of the binomial random variable (BRV). In the BRV's case, if the sample is of size $n$ and we consider each item in the sample as random variable of its own, then $X = X_1+X_2+...+X_n$. To find the $E[X]$, we simply add the $X_i$. Since an item is returned after it is checked, the probability of success does not change. In the case of the HRV, I should expect the probability of success to change because it is not returned back to the population. However, this doesn't seem to be the case. This is my problem.

$\endgroup$
2
$\begingroup$

We have $N$ balls, of which $r$ are red. We draw balls sequentially, without replacement.

Imagine the balls are distinct, they have ID numbers written on them, if you wish with invisible ink. Imagine also drawing all $N$ balls, one after the other. Then all of the sequences of ball ID's are equally likely. So the probability a particular ball is drawn $k$-th is $\frac{1}{N}$, and therefore the probability it is red is $\frac{r}{N}$.

Now let us suppose that we only draw $n$ balls. For $i=1$ to $n$, let random variable $X_i$ be $1$ if the $i$-th drawn ball is red, and $0$ otherwise. Then the number $Y$ of red balls is given by $Y=X_1+\cdots +X_n$. By the linearity of expectation, it follows that $$E(Y)=nE(X_1)=n\Pr(X_1=1)=n\cdot \frac{r}{N}.$$

$\endgroup$
  • 1
    $\begingroup$ My problem is why is the probability of drawing a red ball in the $k$-th trial equal to that drawn in the $k+1$-th trial. I still don't get it why the expected value of every trial is constant and equal to pos $\frac{r}{N}$. Why is $Pr(X_1=1)=\frac{r}{N}$ and $Pr(X_2=1)=\frac{r}{N}$ ? $\endgroup$ – horemheb Jul 9 '16 at 1:08
  • $\begingroup$ Any ball is equally likely to be drawn $k$-th. One could do this the hard way, by tracing all possible histories up to the $k$-th draw. This is a mildly messy conditional probability calculation. However, if we think of the balls as distinct, it is clear that ball with license plate number $6$ is just as likely to be drawn $k$-th as ball with license plate number $17$. $\endgroup$ – André Nicolas Jul 9 '16 at 1:12
  • 1
    $\begingroup$ @horemheb: You are confusing the unconditional probability that (say) the second ball is red with the conditional probability that the second is red, given the first was red. The conditional probability is $\frac{r-1}{N-1}$. However, the unconditional probability, given no information about the result of the first draw, is $\frac{r}{N}$. $\endgroup$ – André Nicolas Jul 9 '16 at 2:20
  • $\begingroup$ I think you're absolutely right. I was subconsciously thinking of the conditional probability all the time. In general, it seems that for a given problem, the probability calculated with the pmf of the hypergeometric dist. is larger than that calculated with the pmf of the binomial dist. If that is so, why is that ? $\endgroup$ – horemheb Jul 9 '16 at 2:47
  • $\begingroup$ If the number of reds and of blues is greater than the number $n$ of draws, then the probabilities can't all be larger, since the number of drawn reds then ranges in each case from $0$ to $n$, and probabilities sum to $1$. Thus in that case some probabilities are larger, some are smaller. However, if $r$ is small compared to $n$, one should expect hypergeometric probabilities to be larger. By the way, one important difference between the binomial and the hypergeometric is that in the geometric case the indicator random variables are independent, while in the hypergeometric case they are not. $\endgroup$ – André Nicolas Jul 9 '16 at 3:07
3
$\begingroup$

Here's an intuition (inspired by @André Nicolas' comment on the accepted answer):

CURRENT INTUITION FOR BINOMIAL: First of all let's question the intuition of the Binomial case. Let's say we have an urn with N balls, whereas R are red balls and N-R white. We want to draw red balls (successes). And we draw n times. The random variable X denotes the number of red balls we draw.

Currently your mental picture of drawing is:

  1. close your eyes
  2. draw (without looking)
  3. open your eyes; look at the ball ("ahaa it's red/white")
  4. put it back (since it's sampling with replacement)
  5. repeat n-1 times

For all draws i in {1, ... n} we have: E[Xi ] = R / N. With linearity of expectation we get E[X] = n * R / N.

NEW INTUITION FOR BINOMIAL: So far so good. But now let's change your mental picture of the Binomial case. Just skip the "looking at the drawn ball":

  1. close your eyes
  2. draw (without looking)
  3. don't open your eyes, but take a picture of the ball (e.g. with your phone) to look at it later
  4. put it back (since it's sampling with replacement)
  5. repeat n-1 times

Here, during the drawing process you have no information about the drawn balls. Only when you look at the pictures later, you will know. Again, E[X] = n * R / N. In the binomial case, there is no difference, if you look at the drawn balls or not. BUT in the hypergeometrical case it does make a difference. So here is the mental picture:

NEW INTUITION FOR HYPERGEOMETRIC:

  1. close your eyes
  2. draw (without looking)
  3. keep the ball in a box (for you to look at it later; eyes still closed)
  4. repeat n-1 times

So you haven't looked at the balls yet. Now, what's the probability that the first ball is a red one? That's easy, it's R / N. You have R successes and draw them from N balls uniformly.

Now, what about the probability, that the second ball (let's say you have it in your hand, eyes closed) is red (You still haven't looked at the first ball yet): Ahaa, it's R/N again. Why? Because you have N unknowns (N-2 in the urne, 1 in the box and 1 in your hand) and still R successes (most of them in the urne, and maybe one in the box and/or in your hand - you don't know where these R balls are). The same is true for the third draw, etc. Since you don't know which ones you have drawn already, the probability of a red ball in your hand does not change.

So again E[Xi] = R/N. With linearity of expectation, we get E[X] = n * R / N (which is, what we were looking for).

This is, what @André Nicolas meant in his comment. There's a difference between P["second ball red"] and P["second ball red" | "first ball red"] (respectively P["second ball red" | "first ball white"]).

Hope that helps

$\endgroup$
1
$\begingroup$

We have a population of $N$ balls, or which $R$ are red, and extract a sample of $n$ balls (without replacement).

Suppose we line the balls in a row as we extract them.   Let $X_k$ be the indicator that the $k$-th ball drawn is red.   That means: $X_k=1$ if it is, $X_k=0$ if it is not.   It is a Bernouli random variable.

Now the expectation of this random variable is the probability that the $k$-th balls is red. $$\begin{align}\mathsf E(X_k)=&~1\cdot\mathsf P(X_k=1)\color{silver}{+0\cdot\mathsf P(X_k0)}\\ =&~ \tfrac RN \end{align}$$

The count of red balls withing the sample is $\sum_{k=1}^n X_k$.   Of course the random variables, $\{X_k\}_{k\in\{1,..,n\}}$, are not independent, but here's a fun fact:

The Linearity of Expectation holds regardless of whether the random variables are independent or not.

Hence we have that:

$$\mathsf E\left(\sum_{k=1}^n X_k\right) ~=~ \sum_{k=1}^n\mathsf E(X_k) ~=~ n\cdot\frac{R}{N}$$

$\endgroup$
  • $\begingroup$ I still don't get it why $Pr(X_1=1)=\frac{r}{N}$ and $Pr(X_2=1)=\frac{r}{N}$ and so on. If we know that $X_1=1$ then why is $Pr(X_2=1)$ still $\frac{r}{N}$. Why is it not $\frac{r-1}{N-1}$ ? $\endgroup$ – horemheb Jul 9 '16 at 1:28
  • 1
    $\begingroup$ @horemheb Indeed, that is the conditional probability $\Pr (X_2=1\mid X_1=1)= \frac{R-1}{N-1}$. Also $\Pr(X_2=1\mid X_1=0) = \frac{R}{N-1}$ and so by the Law of Total Probability the marginal probability is: $$\begin{align}\Pr(X_2=1)~=&~ \Pr(X_2=1\mid X_1=1)\cdot\Pr(X_1=1)+\Pr(X_2=1\mid X_1=0)\cdot\Pr(X_1=0) \\[1ex] =&~ \frac{R-1}{N-1}\cdot\frac{R}{N}+\frac{R}{N-1}\cdot\frac{N-R}{N} \\[1ex] =&~ \frac{R}{N}\end{align}$$ $\endgroup$ – Graham Kemp Jul 9 '16 at 3:30
1
$\begingroup$

The probability of success is indeed the same on every trial.

Say you have four red marbles and six black marbles in an urn, and you sample eight of them without replacement. Suppose "success" means getting a black marble. Then the probability of success on the first trial is $6/10$. What is the probability of success on the second trial? Here's one way to look at that: \begin{align} & \Pr(\text{success on 2nd trial}) \\[6pt] = {} & \Pr\Big((\text{success on 1st trial and success on 2nd}) \text{ or } (\text{failure on 1st and success on 2nd})\Big) \\ = {} & \Pr(\text{success on 1st})\cdot\Pr(\text{success on 2nd} \mid \text{success on 1st}) \\ & {} + \Pr(\text{failure on 1st}) \cdot \Pr(\text{success on 2nd} \mid \text{failure on 1st}) \\[6pt] = {} & \frac 6 {10}\cdot \frac 5 9 + \frac 4 {10} \cdot \frac 6 9 = \frac{54}{90} = \frac{6\times 9}{10\times 9} = \frac 6 {10}. \end{align}

However, it's easier to see by symmetry: just interchange the roles of "1st" and "2nd" above, and see that no numbers change, using things like $$\Pr(\text{success on 1st trial}\mid \text{success on 2nd trial}) = \frac 5 9,$$ etc.

$\endgroup$
  • $\begingroup$ I don't know why but I am surprised by this observation. If the probability of success is not changing, what then separates the Binomial variable from the hypergeometric variable ? I think your answer has put me in the direction that can lead to the source of my confusion. $\endgroup$ – horemheb Jul 9 '16 at 2:02
  • $\begingroup$ @horemheb : What separates them is that with the binomial distribution, the events of success on different trials are independent. With the hypergeometric distribution, they are negatively correlated. The hypergeometric distribution has a smaller variance. Notice that in the example above, if you have a total of $8$ trials, the possible numbers of successes are only $4$, $5$, and $6$, whereas in a binomial experiment with $8$ trials, the number of successes could be any of nine numbers: $0,1,2,3,4,5,6,7,8$. So this is quite different from a binomial distribution. $\qquad$ $\endgroup$ – Michael Hardy Jul 9 '16 at 17:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.