I'm reading a paper: Bect et al 2012, Sequential design of computer experiments for the estimation of a probability of failure. I'm looking at page 2, but the details of the paper don't matter. What matters is that $\alpha\in(0,1)$, and we have an unbiased estimator $$\alpha_m = \frac1m \sum_{i=1}^m \mathbf 1_{A_i}$$

where $\mathbb1_{A_i}$ is an indicator function for the event $A_i$. The authors assert that $$\mathbb E [(\alpha_m-\alpha)^2)]=\frac1m\alpha(1-\alpha)$$ But when I try to work this out for myself, I instead get $\mathbb E [(\alpha_m-\alpha)^2)]= \frac1m \alpha (1-m\alpha)$, which I know is wrong. My reasoning is below. I would appreciate either an explanation for how the authors get their answer, or a diagnosis of what is wrong with my reasoning.

\begin{align*} \mathbb E [(\alpha_m-\alpha)^2)]&=\mathbb E [\alpha_m^2-2\alpha\alpha_m+\alpha)^2]\\ &= \mathbb E[\alpha_m^2] - \alpha^2 \end{align*}

\begin{align} \mathbb E[\alpha_m^2] &= \mathbb E \left[ \frac1{m^2} \left(\sum_{i=1}^m \mathbb 1_{A_i}\right) \right]\\ &= \frac 1{m^2} \mathbb E\left[ \sum_{i=1}^m \mathbb1_{A_i} + \sum_{j=1}^{m\cdot a_m-1} j \right]\\ &=\frac 1m\alpha + \frac 1{2m^2}\mathbb E[(m\alpha_m-1)(m\alpha_m)]\\ &=\frac 1m\alpha + \frac1{2}\mathbb E[\alpha_m^2] - \frac1{2m}\alpha\\ &=\frac 1{2m}\alpha + \frac1{2}\mathbb E[\alpha_m^2]\\ \end{align} which implies that $\mathbb E[\alpha_m^2] = \frac1m \alpha$, which implies that $\mathbb E [(\alpha_m-\alpha)^2)]= \frac1m \alpha (1-m\alpha)$.


Notice that $\sum_{i=1}^m \mathbb{1}_{A_i} \sim Binomial(m, \alpha)$

$$Var\left(\sum_{i=1}^m \mathbb{1}_{A_i}\right)=m\alpha(1-\alpha)$$

\begin{align}\mathbb{E} [(\alpha_m-\alpha)^2]&=Var(\alpha_m)\\&=Var\left(\frac1m\sum_{i=1}^m \mathbb{1}_{A_i}\right) \\ &=\frac1{m^2}m\alpha(1-\alpha) \\&=\frac1m\alpha(1-\alpha)\end{align}

Alternatively, doing it similar to your way:

\begin{align}\mathbb{E}[\alpha_m^2]&=\mathbb{E}\left[ \frac{1}{m^2}\left( \sum_{i=1}^m \mathbb{1}_{A_i}\right)^2\right] \\ &=\mathbb{E}\left[ \frac{1}{m^2}\left( \sum_{i=1}^m \mathbb{1}_{A_i}+2\sum_{i=1}^m\sum_{j<i}^m\mathbb{1}_{A_i \cap A_j}\right)\right]\\&=\frac1{m^2} \left(m\alpha+2\binom{m}{2}\alpha^2\right)\\ &=\frac1m\left( \alpha +(m-1)\alpha^2\right)\\ &=\frac1m\left(\alpha-\alpha^2 \right)+\alpha^2\end{align}

Hence \begin{align}\mathbb{E} \left( \alpha_m^2\right)-\alpha^2&= \frac1m\alpha(1-\alpha)\end{align}

  • $\begingroup$ Thank you -- however, how do we have that $\sum_{i=1}^m \mathbf 1_{A_i}\sim \mathrm{Bin}(m,a)$? The $A_i$ in question are neither independent nor identically distributed. $\endgroup$ – Ceph Jun 17 '17 at 0:02
  • $\begingroup$ $A_i$ refers to $\{ f(X_i) > u \}$ where $X_i$ are independent random variables with distribution $P_X$. They are indeed independent and identically distributed isn't it? $\endgroup$ – Siong Thye Goh Jun 17 '17 at 0:15
  • $\begingroup$ @SiongThyeGoh is correct, and in fact, without the assumption that the $A_i$ are iid, the result does not hold. $\endgroup$ – David Foley Jun 18 '17 at 18:11
  • $\begingroup$ Ah - I missed that the $X_i$ are iid, I was thinking of them as being the points of a LHC design space for some reason. I guess the details of the paper were important after all! Thanks. $\endgroup$ – Ceph Jun 19 '17 at 12:34

You do omit the square as Siong Thye Goh points out, but I believe that's just a typo, as the next line indicates that you intended to consider all the terms in the product.

Your real mistake is

$$\mathbb{E}\Bigg\lbrack \frac{1}{m^2}\Big(\sum_{i=1}^{m}\mathbb{1}_{A_i}\Big)^2\Bigg\rbrack = \frac{1}{m^2}\mathbb{E}\Bigg\lbrack\sum_{i=1}^{m}\mathbb{1}_{A_i} + \sum_{j=1}^{ma_m -1}j\Bigg\rbrack$$

because each of the cross terms (the terms in the rightmost sum) occur twice. So:

$$\begin{equation} \begin{split} \mathbb{E}\lbrack a_m^2\rbrack = \mathbb{E}\Bigg\lbrack \frac{1}{m^2}\Big(\sum_{i=1}^{m}\mathbb{1}_{A_i}\Big)^2\Bigg\rbrack & = \frac{1}{m^2}\mathbb{E}\Bigg\lbrack\sum_{i=1}^{m}\mathbb{1}_{A_i} + 2\sum_{j=1}^{ma_m -1}j\Bigg\rbrack \\ & = \frac{1}{m}a + \frac{1}{m^2}\mathbb{E}\lbrack(ma_m - 1)(ma_m)\rbrack \\ & = \frac{1}{m}a + \mathbb{E}\lbrack a_m^2\rbrack - \frac{1}{m}a \\ & = \mathbb{E}\lbrack a_m^2\rbrack \end{split} \end{equation}$$

A rather disappointing result.

  • $\begingroup$ Yikes, you are correct. Thank you. $\endgroup$ – Ceph Jun 17 '17 at 0:04
  • $\begingroup$ Thanks for pointing out that that is just a typo, I stopped reading too early. $\endgroup$ – Siong Thye Goh Jun 17 '17 at 0:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.