Assume that I have a binary distribution with probability $p$ determined by the previous outcome. For instance, for random variable $X$, define it's initial distribution to be $X_0 \sim Bernoulli(p_0) $ and subsequent values of $p$ to be $ (\forall\ n \in \Bbb N)$:$$ p_{n+1} = \left\{ \begin{array}{c} f(p_n)\qquad (X_n=0) \\ g(p_n)\qquad (X_n=1) \\ \end{array} \right. $$ and $X_n \sim Bernoulli(p_n) $.

Is there a way to determine if the distribution is stable based on $f$ and $g$? If it's stable, is it possible to describe the stable distribution of $X$ (or its characteristics) via $f$ and $g$, possibly using a Markov Chain?

In a broader sense, are there methods of describing this type of distribution where the next sampled value is generated from a varying probability distribution, which depends on the current sampled value?

  • $\begingroup$ It looks like a Markov process. $\endgroup$ – herb steinberg Dec 24 '19 at 0:52
  • $\begingroup$ I imagine for general $f,g$ it is pretty hopeless. Yes it is Markov, but if $f,g$ are completely arbitrary one can imagine e.g. $f(f(g(g(p_0)))) \neq f(g(f(g(p_0))))$ and so on...i.e. the $16$ possible ways to make a chain of $4$ functions would result in $16$ different $p_4$ values. For such "bad-behaving" $f,g$ after $n$ steps you would need $2^n$ states just to account for the $2^n$ possible different values of $p_n$. $\endgroup$ – antkam Dec 24 '19 at 5:08

I'm not sure I understand what "stable distribution" means. But I'll attempt to share some of my understanding on this problem.

Through your equations, you seem to have described a sequence $p_0, p_1, p_2... $ of real numbers. For any given $f, g$, one might ask whether this sequence converges to a limit. Of course, this depends on what $f$ and $g$ are. For instance, we can define $f(p) = g(p) = 1 - p$. If we start off with $p_0 \neq \frac{1}{2}$, then this sequence alternates between $p_0$ and $1-p_0$ and it does not converge. The convergence is trivial if $p_0=\frac{1}{2}$.

Even if the sequence does not converge, it is possible that the values it takes are from a particular set of values (state space). This is because the process of generating $p_i$'s is essentially a Markov Chain. The state space, however, depends on the range of $f$ and $g$. In the example above, the state space of the Markov Chain is $\{p_0, 1-p_0 \}$. This state space might well be finite, countably infinite or even uncountably infinite! Of course, $p_n$ will be able to take on only one of a countable set of values (see comment by @antkam), but this is conditional on the fact that $p_0$ is given. If $p_0$ is unknown, then in the most general case, the state space is $[0,1]$ which is uncountable.

Let's say that the range is countable and finite and you index this using the function $I$ (i.e., $I(0), I(1), ...$ are the possible states). Then, you can define the state transition matrix $M_i$ for each stage $i$ such that $M_i (x, y)$ is the probability that $p_{i}=I(x)$ and $p_{i+1} = I(y)$, for all $x,y \in domain(I)$. Then you'll need to find a way of checking whether the sequence of vectors $Q_i = (\prod_{k=1}^{i}M_k)v$ converges, where $v$ is the vector satisfying $v[k] = Pr(I(k) = p_0)$. In Markov Chain terminology, we are trying to decide if the distribution is $stationary$.

As an exercise, you can try doing it for the example we had above. It should be very easy.

I hope this helps. Feel free to ask any questions about this. :)

| cite | improve this answer | |
  • $\begingroup$ +1 for nice explanations. A minor technical detail: Suppose $p_0$ is given. Then for any specific $n$, $p_n$ can have at most $2^n$ possible values. This set is not only countable, but finite. However, if you take all $n$, i.e. consider the set $\{p_n: n \in \mathbb{N}\}$, then I think (not 100% sure) this set can be uncountably infinite. Anyway, this is a minor technical detail. $\endgroup$ – antkam Dec 25 '19 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.