# General state space Markov chain $X_{n+1}$ with piecewise linear recursion.

I have sequence of random variables defined by the following recursion:

$$X_{n+1} = X_n+\begin{cases} \alpha(S_n - X_n), \text{ if } S_n > X_n \\ \beta(S_n - X_n), \text{ if } S_n < X_n, \end{cases}$$ where $$0<\beta < \alpha <1$$ are constants, $$(S_n)$$ are i.i.d with known distributions. Also, $$S_n$$ independent of $$\sigma(X_1, X_2,\dots, X_n)$$ and $$X_ 0 = 0.$$

Initially, I asked about the convergence/limiting distribution of $$X_n,$$ but after doing some research, I realize that it is generally considered a very difficult problem - to obtain explicit distribution/asymptotics.

Therefore, I want to ask following questions with increasing orders of difficulties (according to my very limited probability theory knowledge.)

1) Can we at least prove that it has a limiting distribution? It looks like one can formulate this as a general state space Markov Chain but there do not seem to be an abundance of sources on this topic. Probability by Durrett has a brief chapter on it and he mentions that discrete Orstein- Uhlehnbeck process: $$V_{n+1} = \theta V_n+\xi_n$$ is an example of a discrete time, general state space Markov Chain. However, most of the resources I could find on the internet refers to the continuous one and as such my hope of modifying proofs for OU did not pan out.

2) If there is a limiting distribution, what kind of qualitative results can I hope to achieve? For example, one has the following for the expected value: $$\mathbb{E}[X_{n+1}] = \mathbb{E}[X_n](1 - \beta ) + \beta\mu + ( \alpha - \beta)\mathbb{E}[\delta_n\mathbb{1}_{\delta_n >0}],$$ where $$\delta_n = S_n - X_n,$$ and $$\mu = \mathbb{E}[R_n].$$ But then, the issue I am having is manipulating: $$P(S_n - X_n > t|S_n > X_n)$$, which will come from the last term.

I will greatly appreciate if anyone has some ideas or point me to a helpful source.

Simulation: I attach some simulations that seem to suggest that there is a bounded, limiting distribution. ## This question has an open bounty worth +250 reputation from dezdichado ending in 5 days.

This question has not received enough attention.

• I think it's a Markov Chain. Why wouldn't it be? It is certainly true that $X_{n+1}$ is dependent on $X_{n-1}$ ("implicitly" as you said), but the Markov property is that such a dependence disappears when we also condition on $X_n$. I.e., if you know $X_n$, then also knowing $X_{n-1}$ gives you no additional info w.r.t. distribution of $X_{n+1}$, which is the case here. – antkam Nov 7 at 3:53
• @antkam I see that makes sense. – dezdichado Nov 7 at 4:47
• @antkam OP said "$S_n$ is independent of $X_n$", so it may very well be that $S_n$ is not independent of $X_{n-1}$. – mathworker21 yesterday
• @mathworker21 - oh you are right! I had assumed $S_n$ is independent of all $X_j$'s. But if $S_n$ depends on $X_{n-1}$ then this is indeed not a Markov Chain. Maybe the OP can clarify? – antkam yesterday
• sorry, I should clarify $S_n$ is independent of $\sigma(X_1, ...,X_n).$ – dezdichado yesterday