3
$\begingroup$

enter image description here

How is the variance of Bernoulli distribution derived from the variance definition?

$\endgroup$
  • $\begingroup$ use the alternative definition of variance: $\mathbf{E}X^2 - (\mathbf{E}X)^2$ $\endgroup$ – Alex Jun 17 '14 at 14:40
1
$\begingroup$

PMF of the Bernoulli distribution is $$ p(x)=p^x(1-p)^{1-x}\qquad;\qquad\text{for}\ x\in\{0,1\}, $$ and the $n$-moment of a discrete random variable is $$ \text{E}[X^n]=\sum_{x\,\in\,\Omega} x^np(x). $$ Let $X$ be a random variable that follows a Bernoulli distribution, then \begin{align} \text{E}[X]&=\sum_{x\in\{0,1\}} x\ p^x(1-p)^{1-x}\\ &=0\cdot p^0(1-p)^{1-0}+1\cdot p^1(1-p)^{1-1}\\ &=0+p\\ &=p \end{align} and \begin{align} \text{E}[X^2]&=\sum_{x\in\{0,1\}} x^2\ p^x(1-p)^{1-x}\\ &=0^2\cdot p^0(1-p)^{1-0}+1^2\cdot p^1(1-p)^{1-1}\\ &=0+p\\ &=p. \end{align} Thus \begin{align} \text{Var}[X]&=\text{E}[X^2]-\left(\text{E}[X]\right)^2\\ &=p-p^2\\ &=\color{blue}{p(1-p)}, \end{align} or \begin{align} \text{Var}[X]&=\text{E}\left[\left(X-\text{E}[X]\right)^2\right]\\ &=\text{E}\left[\left(X-p\right)^2\right]\\ &=\sum_{x\in\{0,1\}} (x-p)^2\ p^x(1-p)^{1-x}\\ &=(0-p)^2\ p^0(1-p)^{1-0}+(1-p)^2\ p^1(1-p)^{1-1}\\ &=p^2(1-p)+p(1-p)^2\\ &=(1-p)(p^2+p(1-p)\\ &=\color{blue}{p(1-p)}. \end{align}

$\endgroup$
  • 1
    $\begingroup$ Remember, support of the Bernoulli distribution is only two, i.e. $0$ and $1$. $\endgroup$ – Tunk-Fey Jun 17 '14 at 15:13
  • $\begingroup$ Why does the variance above in my question has $E[X^2] - (E[X])^2$ at the end of the calculation? Why $2E[X]E[X]$ is canceled out? $\endgroup$ – user122358 Jun 17 '14 at 15:14
  • 1
    $\begingroup$ @user122358 Wiki explanation is already obvious. Expectation only works for random variable, in the example $X$ and $E[X^n]$ is a constant. Then using property of expectation, we have $E[aX]=a\cdot E[X]$, where $a$ is a constant. $\endgroup$ – Tunk-Fey Jun 17 '14 at 15:17
  • 1
    $\begingroup$ @user122358 from the third line Wiki explanation \begin{align} Var[X]&=E[X^2]-2E[X]\cdot E[X]+(E[X])^2\\ &=E[X^2]-2(E[X])^2+(E[X])^2\\ &=E[X^2]-(E[X])^2. \end{align} $\endgroup$ – Tunk-Fey Jun 17 '14 at 15:20
0
$\begingroup$

IF $X \sim$ Bernoulli($p$), then what's the distribution of $X^2$? Find it, then find its expectation, that'll be $\mathbf{E}X^2$, and you already have $(\mathbf{E}X)^2$.

EDIT: the way it's done in the example you gave, it seems that they defined a different rv: $W = (X- \mathbf{E}X)^2$, which takes values $(0-p)^2$ and $(1-p)^2$ with corresponding probabilities.

$\endgroup$
  • $\begingroup$ I am afraid that I am not really with you... $\endgroup$ – user122358 Jun 17 '14 at 14:42
  • $\begingroup$ pls see the edit $\endgroup$ – Alex Jun 17 '14 at 14:43
0
$\begingroup$

In a Bernoulli distribution, $E(X) = p$. Remember, the Bernoulli takes value $1$ with probability $p$ and $0$ with probability $1-p$, so the expectation of $X$ is $1\cdot p + 0\cdot (1-p) = p$. The expectation of any function of the Bernoulli is similarly $(E(g(x)) = g(1)\cdot p + g(0)\cdot(1-p)$ Thus $$ \begin{align} Var(X) &= E\left((X - E(X))^2\right)\\ &= \overbrace{\underbrace{(1-p)^2}_{g(1)}\cdot p}^\textrm{Bernoulli = 1; probability p} + \underbrace{\overbrace{(0-p)^2}^{g(0)}\cdot(1-p)}_{\textrm{Bernoulli = 0; probability (1-p)}}\\ \end{align} $$ The rest follows what you brought in the question

$\endgroup$
  • $\begingroup$ What is it concatenated with $+$ sign? I am too new to this. $\endgroup$ – user122358 Jun 17 '14 at 14:54
  • $\begingroup$ I'm sorry, I don't understand your question. If you mean why the + sign, remember what the definition of a Bernoulli expectation is (I brought it in the second and third sentence of the answer), it is the value of whatever when the Bernoulli is 1 times the probability of the Bernoulli being 1 plus the value of whatever when the Bernoulli is 0 times the probability of the Bernoulli being 0. $\endgroup$ – Avraham Jun 17 '14 at 14:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.