Suppose $Y$ is a Bernoulli random variable s.t. Prob$\{Y=1\} = \alpha\in(0,1)$. Further suppose I observe a sequence $\{X_t\}_{t=1}^\infty$ of i.i.d. Bernoulli random variables s.t. $\text{Prob} \{X_t=1|Y = y\} = \beta_y\in(0,1)$ $\forall y\in\{0,1\}$.

How would I calculate $\underset{t\to\infty}{\text{plim}}\ \text{Prob}\Big\{Y=1|\{X_{\tau}\}_{\tau=1}^t\ \Big\}?$ Thank you very much in advance!

P.S. If there are conditions (on $\alpha,\beta_0,\text{ and }\beta_1$) under which the above plim may not exist, I have a followup question. (But I suppose we'll cross that bridge when/if we get there!!) Thanks again!

  • $\begingroup$ A simple remark if you add the hypothesis that $Y$ is conditionally independent of $X_t \forall t$ then $\underset{t\to\infty}{\text{plim}}\ \text{Prob}\Big\{Y=1|\{X_{\tau}\}_{\tau=1}^t\ \Big\} =\text{Prob}\Big\{Y=1\Big\} = \alpha$ $\endgroup$
    – TheBridge
    Mar 6 at 13:09

Using Bayes' Theorem and the Law of Total Probability we have: $$\textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \frac{\textrm{Prob}\left\{\left\{X_\tau\right\}_{\tau=1}^t \mid Y = 1\right\}\textrm{Prob}\left\{Y = 1\right\}}{\textrm{Prob}\left\{\left\{X_\tau\right\}_{\tau=1}^t \mid Y = 1\right\}\textrm{Prob}\left\{Y = 1\right\} + \textrm{Prob}\left\{\left\{X_\tau\right\}_{\tau=1}^t \mid Y = 0\right\}\textrm{Prob}\left\{Y = 0\right\}}.$$ Let $S_t := \sum_{\tau=1}^t X_\tau$. From your description we have $$\textrm{Prob}\left\{\left\{X_\tau\right\}_{\tau=1}^t \mid Y = y\right\} = \beta_y^{S_t} \left(1-\beta_y\right)^{t-S_t}.$$ Hence $$\textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \frac{\beta_1^{S_t} \left(1-\beta_1\right)^{t-S_t}\alpha}{\beta_1^{S_t} \left(1-\beta_1\right)^{t-S_t}\alpha + \beta_0^{S_t} \left(1-\beta_0\right)^{t-S_t}(1-\alpha)}.$$ If $\beta_0 = \beta_1$ then this probability will equal $\alpha$ no matter what data we observe. Otherwise, let $$R_t = \frac{\beta_0^{S_t} \left(1-\beta_0\right)^{t-S_t}}{\beta_1^{S_t} \left(1-\beta_1\right)^{t-S_t}}.$$ Then $$\textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \frac{\alpha}{\alpha + R_t(1-\alpha)}.$$ Now we need to work out what $R_t$ converges to. We first note that $S_t = t\bar{X}_t$. Given $Y = y$, we know that $\underset{t\to\infty}{\text{plim}}\ \bar{X}_t = \beta_y$.

Considering the case $Y = 1$, $$\underset{t\to\infty}{\text{plim}}\ R_t = \lim_{t\to\infty} \frac{\beta_0^{\beta_1t} \left(1-\beta_0\right)^{t\left(1-\beta_1\right)}}{\beta_1^{\beta_1t} \left(1-\beta_1\right)^{t\left(1-\beta_1\right)}} = \lim_{t\to\infty} \left[\frac{\beta_0^{\beta_1} \left(1-\beta_0\right)^{\left(1-\beta_1\right)}}{\beta_1^{\beta_1} \left(1-\beta_1\right)^{\left(1-\beta_1\right)}}\right]^t.$$ Using differentiation you can see that $x^{\beta_1}(1-x)^{1-\beta_1}$ is maximised when $x = \beta_1$. So $$\beta_0^{\beta_1} \left(1-\beta_0\right)^{\left(1-\beta_1\right)} < \beta_1^{\beta_1} \left(1-\beta_1\right)^{\left(1-\beta_1\right)}$$ and hence $$\underset{t\to\infty}{\text{plim}}\ R_t = \lim_{t\to\infty} \left[\frac{\beta_0^{\beta_1} \left(1-\beta_0\right)^{\left(1-\beta_1\right)}}{\beta_1^{\beta_1} \left(1-\beta_1\right)^{\left(1-\beta_1\right)}}\right]^t = 0,$$ and so $$\underset{t\to\infty}{\text{plim}}\ \textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \underset{t\to\infty}{\text{plim}}\ \frac{\alpha}{\alpha + R_t(1-\alpha)} = 1.$$

Considering the case $Y=0$, we can use similar reasoning to see that $$\underset{t\to\infty}{\text{plim}}\ R_t = \infty$$ and so $$\underset{t\to\infty}{\text{plim}}\ \textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \underset{t\to\infty}{\text{plim}}\ \frac{\alpha}{\alpha + R_t(1-\alpha)} = 0.$$

In conclusion,

  • if $\beta_0 = \beta_1$ then $\underset{t\to\infty}{\text{plim}}\ \textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = \alpha$.
  • Otherwise, if $Y = 1$ then $X_{\tau} \sim \textrm{Bernoulli}(\beta_1)$ and $\underset{t\to\infty}{\text{plim}}\ \textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = 1$.
  • And if $Y = 0$ then $X_{\tau} \sim \textrm{Bernoulli}(\beta_0)$ and $\underset{t\to\infty}{\text{plim}}\ \textrm{Prob}\left\{Y = 1 \mid \left\{X_\tau\right\}_{\tau=1}^t\right\} = 0$.
  • 1
    $\begingroup$ Thanks so very much, Alex!! I really appreciate how thorough and clear your explanation is. $\endgroup$ Mar 9 at 4:56
  • $\begingroup$ You're very welcome! :) $\endgroup$
    – Alex
    Mar 9 at 7:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.