# Calculating the plim of a posterior belief about a Bernoulli r.v. formed from observing an infinite sequence of other Bernoulli r.v.'s

Suppose $$Y$$ is a Bernoulli random variable s.t. Prob$$\{Y=1\} = \alpha\in(0,1)$$. Further suppose I observe a sequence $$\{X_t\}_{t=1}^\infty$$ of i.i.d. Bernoulli random variables s.t. $$\text{Prob} \{X_t=1|Y = y\} = \beta_y\in(0,1)$$ $$\forall y\in\{0,1\}$$.

How would I calculate $$\underset{t\to\infty}{\text{plim}}\ \text{Prob}\Big\{Y=1|\{X_{\tau}\}_{\tau=1}^t\ \Big\}?$$ Thank you very much in advance!

P.S. If there are conditions (on $$\alpha,\beta_0,\text{ and }\beta_1$$) under which the above plim may not exist, I have a followup question. (But I suppose we'll cross that bridge when/if we get there!!) Thanks again!

• A simple remark if you add the hypothesis that $Y$ is conditionally independent of $X_t \forall t$ then $\underset{t\to\infty}{\text{plim}}\ \text{Prob}\Big\{Y=1|\{X_{\tau}\}_{\tau=1}^t\ \Big\} =\text{Prob}\Big\{Y=1\Big\} = \alpha$ Mar 6 at 13:09

Using Bayes' Theorem and the Law of Total Probability we have: $$\textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \frac{\textrm{Prob}\left\{\left\{X_\tau\right\}_{\tau=1}^t \mid Y = 1\right\}\textrm{Prob}\left\{Y = 1\right\}}{\textrm{Prob}\left\{\left\{X_\tau\right\}_{\tau=1}^t \mid Y = 1\right\}\textrm{Prob}\left\{Y = 1\right\} + \textrm{Prob}\left\{\left\{X_\tau\right\}_{\tau=1}^t \mid Y = 0\right\}\textrm{Prob}\left\{Y = 0\right\}}.$$ Let $$S_t := \sum_{\tau=1}^t X_\tau$$. From your description we have $$\textrm{Prob}\left\{\left\{X_\tau\right\}_{\tau=1}^t \mid Y = y\right\} = \beta_y^{S_t} \left(1-\beta_y\right)^{t-S_t}.$$ Hence $$\textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \frac{\beta_1^{S_t} \left(1-\beta_1\right)^{t-S_t}\alpha}{\beta_1^{S_t} \left(1-\beta_1\right)^{t-S_t}\alpha + \beta_0^{S_t} \left(1-\beta_0\right)^{t-S_t}(1-\alpha)}.$$ If $$\beta_0 = \beta_1$$ then this probability will equal $$\alpha$$ no matter what data we observe. Otherwise, let $$R_t = \frac{\beta_0^{S_t} \left(1-\beta_0\right)^{t-S_t}}{\beta_1^{S_t} \left(1-\beta_1\right)^{t-S_t}}.$$ Then $$\textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \frac{\alpha}{\alpha + R_t(1-\alpha)}.$$ Now we need to work out what $$R_t$$ converges to. We first note that $$S_t = t\bar{X}_t$$. Given $$Y = y$$, we know that $$\underset{t\to\infty}{\text{plim}}\ \bar{X}_t = \beta_y$$.

Considering the case $$Y = 1$$, $$\underset{t\to\infty}{\text{plim}}\ R_t = \lim_{t\to\infty} \frac{\beta_0^{\beta_1t} \left(1-\beta_0\right)^{t\left(1-\beta_1\right)}}{\beta_1^{\beta_1t} \left(1-\beta_1\right)^{t\left(1-\beta_1\right)}} = \lim_{t\to\infty} \left[\frac{\beta_0^{\beta_1} \left(1-\beta_0\right)^{\left(1-\beta_1\right)}}{\beta_1^{\beta_1} \left(1-\beta_1\right)^{\left(1-\beta_1\right)}}\right]^t.$$ Using differentiation you can see that $$x^{\beta_1}(1-x)^{1-\beta_1}$$ is maximised when $$x = \beta_1$$. So $$\beta_0^{\beta_1} \left(1-\beta_0\right)^{\left(1-\beta_1\right)} < \beta_1^{\beta_1} \left(1-\beta_1\right)^{\left(1-\beta_1\right)}$$ and hence $$\underset{t\to\infty}{\text{plim}}\ R_t = \lim_{t\to\infty} \left[\frac{\beta_0^{\beta_1} \left(1-\beta_0\right)^{\left(1-\beta_1\right)}}{\beta_1^{\beta_1} \left(1-\beta_1\right)^{\left(1-\beta_1\right)}}\right]^t = 0,$$ and so $$\underset{t\to\infty}{\text{plim}}\ \textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \underset{t\to\infty}{\text{plim}}\ \frac{\alpha}{\alpha + R_t(1-\alpha)} = 1.$$

Considering the case $$Y=0$$, we can use similar reasoning to see that $$\underset{t\to\infty}{\text{plim}}\ R_t = \infty$$ and so $$\underset{t\to\infty}{\text{plim}}\ \textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \underset{t\to\infty}{\text{plim}}\ \frac{\alpha}{\alpha + R_t(1-\alpha)} = 0.$$

In conclusion,

• if $$\beta_0 = \beta_1$$ then $$\underset{t\to\infty}{\text{plim}}\ \textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \alpha$$.
• Otherwise, if $$Y = 1$$ then $$X_{\tau} \sim \textrm{Bernoulli}(\beta_1)$$ and $$\underset{t\to\infty}{\text{plim}}\ \textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = 1$$.
• And if $$Y = 0$$ then $$X_{\tau} \sim \textrm{Bernoulli}(\beta_0)$$ and $$\underset{t\to\infty}{\text{plim}}\ \textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = 0$$.
• Thanks so very much, Alex!! I really appreciate how thorough and clear your explanation is. Mar 9 at 4:56
• You're very welcome! :)
– Alex
Mar 9 at 7:54