How is the variance of Bernoulli distribution derived from the variance definition?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ use the alternative definition of variance: $\mathbf{E}X^2 - (\mathbf{E}X)^2$ $\endgroup$– AlexJun 17, 2014 at 14:40
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
4
PMF of the Bernoulli distribution is $$ p(x)=p^x(1-p)^{1-x}\qquad;\qquad\text{for}\ x\in\{0,1\}, $$ and the $n$-moment of a discrete random variable is $$ \text{E}[X^n]=\sum_{x\,\in\,\Omega} x^np(x). $$ Let $X$ be a random variable that follows a Bernoulli distribution, then \begin{align} \text{E}[X]&=\sum_{x\in\{0,1\}} x\ p^x(1-p)^{1-x}\\ &=0\cdot p^0(1-p)^{1-0}+1\cdot p^1(1-p)^{1-1}\\ &=0+p\\ &=p \end{align} and \begin{align} \text{E}[X^2]&=\sum_{x\in\{0,1\}} x^2\ p^x(1-p)^{1-x}\\ &=0^2\cdot p^0(1-p)^{1-0}+1^2\cdot p^1(1-p)^{1-1}\\ &=0+p\\ &=p. \end{align} Thus \begin{align} \text{Var}[X]&=\text{E}[X^2]-\left(\text{E}[X]\right)^2\\ &=p-p^2\\ &=\color{blue}{p(1-p)}, \end{align} or \begin{align} \text{Var}[X]&=\text{E}\left[\left(X-\text{E}[X]\right)^2\right]\\ &=\text{E}\left[\left(X-p\right)^2\right]\\ &=\sum_{x\in\{0,1\}} (x-p)^2\ p^x(1-p)^{1-x}\\ &=(0-p)^2\ p^0(1-p)^{1-0}+(1-p)^2\ p^1(1-p)^{1-1}\\ &=p^2(1-p)+p(1-p)^2\\ &=(1-p)(p^2+p(1-p)\\ &=\color{blue}{p(1-p)}. \end{align}
-
1$\begingroup$ Remember, support of the Bernoulli distribution is only two, i.e. $0$ and $1$. $\endgroup$– Tunk-FeyJun 17, 2014 at 15:13
-
$\begingroup$ Why does the variance above in my question has $E[X^2] - (E[X])^2$ at the end of the calculation? Why $2E[X]E[X]$ is canceled out? $\endgroup$ Jun 17, 2014 at 15:14
-
1$\begingroup$ @user122358 Wiki explanation is already obvious. Expectation only works for random variable, in the example $X$ and $E[X^n]$ is a constant. Then using property of expectation, we have $E[aX]=a\cdot E[X]$, where $a$ is a constant. $\endgroup$– Tunk-FeyJun 17, 2014 at 15:17
-
1$\begingroup$ @user122358 from the third line Wiki explanation \begin{align} Var[X]&=E[X^2]-2E[X]\cdot E[X]+(E[X])^2\\ &=E[X^2]-2(E[X])^2+(E[X])^2\\ &=E[X^2]-(E[X])^2. \end{align} $\endgroup$– Tunk-FeyJun 17, 2014 at 15:20
$\begingroup$
$\endgroup$
2
IF $X \sim$ Bernoulli($p$), then what's the distribution of $X^2$? Find it, then find its expectation, that'll be $\mathbf{E}X^2$, and you already have $(\mathbf{E}X)^2$.
EDIT: the way it's done in the example you gave, it seems that they defined a different rv: $W = (X- \mathbf{E}X)^2$, which takes values $(0-p)^2$ and $(1-p)^2$ with corresponding probabilities.
-
$\begingroup$ I am afraid that I am not really with you... $\endgroup$ Jun 17, 2014 at 14:42
-
$\begingroup$
$\endgroup$
2
In a Bernoulli distribution, $E(X) = p$. Remember, the Bernoulli takes value $1$ with probability $p$ and $0$ with probability $1-p$, so the expectation of $X$ is $1\cdot p + 0\cdot (1-p) = p$. The expectation of any function of the Bernoulli is similarly $(E(g(x)) = g(1)\cdot p + g(0)\cdot(1-p)$ Thus $$ \begin{align} Var(X) &= E\left((X - E(X))^2\right)\\ &= \overbrace{\underbrace{(1-p)^2}_{g(1)}\cdot p}^\textrm{Bernoulli = 1; probability p} + \underbrace{\overbrace{(0-p)^2}^{g(0)}\cdot(1-p)}_{\textrm{Bernoulli = 0; probability (1-p)}}\\ \end{align} $$ The rest follows what you brought in the question
-
$\begingroup$ What is it concatenated with $+$ sign? I am too new to this. $\endgroup$ Jun 17, 2014 at 14:54
-
$\begingroup$ I'm sorry, I don't understand your question. If you mean why the + sign, remember what the definition of a Bernoulli expectation is (I brought it in the second and third sentence of the answer), it is the value of whatever when the Bernoulli is 1 times the probability of the Bernoulli being 1 plus the value of whatever when the Bernoulli is 0 times the probability of the Bernoulli being 0. $\endgroup$– AvrahamJun 17, 2014 at 14:56