Calculating the plim of a posterior belief about a Bernoulli r.v. formed from observing an infinite sequence of other Bernoulli r.v.'s Suppose $Y$ is a Bernoulli random variable s.t. Prob$\{Y=1\} = \alpha\in(0,1)$. Further suppose I observe a sequence $\{X_t\}_{t=1}^\infty$ of i.i.d. Bernoulli random variables s.t. $\text{Prob} \{X_t=1|Y = y\} = \beta_y\in(0,1)$ $\forall y\in\{0,1\}$.
How would I calculate $\underset{t\to\infty}{\text{plim}}\ \text{Prob}\Big\{Y=1|\{X_{\tau}\}_{\tau=1}^t\ \Big\}?$ Thank you very much in advance!
P.S. If there are conditions (on $\alpha,\beta_0,\text{ and }\beta_1$) under which the above plim may not exist, I have a followup question. (But I suppose we'll cross that bridge when/if we get there!!) Thanks again!
 A: Using Bayes' Theorem and the Law of Total Probability we have:
$$\textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \frac{\textrm{Prob}\left\{\left\{X_\tau\right\}_{\tau=1}^t \mid Y = 1\right\}\textrm{Prob}\left\{Y = 1\right\}}{\textrm{Prob}\left\{\left\{X_\tau\right\}_{\tau=1}^t \mid Y = 1\right\}\textrm{Prob}\left\{Y = 1\right\} + \textrm{Prob}\left\{\left\{X_\tau\right\}_{\tau=1}^t \mid Y = 0\right\}\textrm{Prob}\left\{Y = 0\right\}}.$$
Let $S_t := \sum_{\tau=1}^t X_\tau$. From your description we have
$$\textrm{Prob}\left\{\left\{X_\tau\right\}_{\tau=1}^t \mid Y = y\right\} = \beta_y^{S_t} \left(1-\beta_y\right)^{t-S_t}.$$
Hence
$$\textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \frac{\beta_1^{S_t} \left(1-\beta_1\right)^{t-S_t}\alpha}{\beta_1^{S_t} \left(1-\beta_1\right)^{t-S_t}\alpha + \beta_0^{S_t} \left(1-\beta_0\right)^{t-S_t}(1-\alpha)}.$$
If $\beta_0 = \beta_1$ then this probability will equal $\alpha$ no matter what data we observe. Otherwise, let
$$R_t = \frac{\beta_0^{S_t} \left(1-\beta_0\right)^{t-S_t}}{\beta_1^{S_t} \left(1-\beta_1\right)^{t-S_t}}.$$
Then
$$\textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \frac{\alpha}{\alpha + R_t(1-\alpha)}.$$
Now we need to work out what $R_t$ converges to. We first note that $S_t = t\bar{X}_t$. Given $Y = y$, we know that $\underset{t\to\infty}{\text{plim}}\ \bar{X}_t = \beta_y$.
Considering the case $Y = 1$,
$$\underset{t\to\infty}{\text{plim}}\ R_t = \lim_{t\to\infty} \frac{\beta_0^{\beta_1t} \left(1-\beta_0\right)^{t\left(1-\beta_1\right)}}{\beta_1^{\beta_1t} \left(1-\beta_1\right)^{t\left(1-\beta_1\right)}} = \lim_{t\to\infty} \left[\frac{\beta_0^{\beta_1} \left(1-\beta_0\right)^{\left(1-\beta_1\right)}}{\beta_1^{\beta_1} \left(1-\beta_1\right)^{\left(1-\beta_1\right)}}\right]^t.$$
Using differentiation you can see that $x^{\beta_1}(1-x)^{1-\beta_1}$ is maximised when $x = \beta_1$. So
$$\beta_0^{\beta_1} \left(1-\beta_0\right)^{\left(1-\beta_1\right)} < \beta_1^{\beta_1} \left(1-\beta_1\right)^{\left(1-\beta_1\right)}$$
and hence
$$\underset{t\to\infty}{\text{plim}}\ R_t = \lim_{t\to\infty} \left[\frac{\beta_0^{\beta_1} \left(1-\beta_0\right)^{\left(1-\beta_1\right)}}{\beta_1^{\beta_1} \left(1-\beta_1\right)^{\left(1-\beta_1\right)}}\right]^t = 0,$$
and so
$$\underset{t\to\infty}{\text{plim}}\ \textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \underset{t\to\infty}{\text{plim}}\ \frac{\alpha}{\alpha + R_t(1-\alpha)} = 1.$$
Considering the case $Y=0$, we can use similar reasoning to see that
$$\underset{t\to\infty}{\text{plim}}\ R_t = \infty$$
and so
$$\underset{t\to\infty}{\text{plim}}\ \textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \underset{t\to\infty}{\text{plim}}\ \frac{\alpha}{\alpha + R_t(1-\alpha)} = 0.$$
In conclusion,

*

*if $\beta_0 = \beta_1$ then $\underset{t\to\infty}{\text{plim}}\ \textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \alpha$.

*Otherwise, if $Y = 1$ then $X_{\tau} \sim \textrm{Bernoulli}(\beta_1)$ and $\underset{t\to\infty}{\text{plim}}\ \textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = 1$.

*And if $Y = 0$ then $X_{\tau} \sim \textrm{Bernoulli}(\beta_0)$ and $\underset{t\to\infty}{\text{plim}}\ \textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = 0$.

