I am kind of stuck on how to proceed on this.

$X_n$ is an IID process with $$f_{X_n}(y)= \frac\lambda2 e^{-\lambda |y|}$$

There is a stationary autoregressive process $Y_n$ defined as $$Y_n=\rho Y_{n-1}+X_n$$

The goal is to find $Pr(Y_n > X_n +1/2)$. Any hints?

Edit: Does the fact that the process is first order Markov and stationary help?

  • $\begingroup$ What is the context of this problem? It looks to me that this probability does not admit an easy expression. $\endgroup$
    – user126540
    Aug 2, 2015 at 9:36
  • $\begingroup$ I came across it in the introductory chapter in Vector Quantization and Signal Compression by Gersho and Gray, which I have taken up for self study. The chapter deals with basic probability concepts and random processes. $\endgroup$
    – Iconoclast
    Aug 2, 2015 at 13:16
  • $\begingroup$ the problem says that htis kind of proceess is popularly used to model speech signals wth a properly chosen value of $\lambda$. $\endgroup$
    – Iconoclast
    Aug 2, 2015 at 13:18

2 Answers 2


First, $$ \mathsf{P}\left(Y_n-X_n>\frac{1}{2}\right)=\mathsf{P}\left(Y_{n-1}>\frac{1}{2\rho}\right) $$

and $Y_n=(1-\rho L)^{-1}X_{n}=\sum_{k=0}^{\infty}\rho^k X_{n-k}$, where $L$ is the lag operator. Then, using the characteristic function of a Laplace r.v.,

$$ \varphi_{\sum_{k=0}^{N}\rho^k X_{n-k}}(t)=\prod_{k=0}^{N}\varphi_{X_{n-k}}\left(\rho^k t\right)=\exp\left\{-\sum_{k=0}^{N}\ln\left(1+\rho^{2k}\frac{t^2}{\lambda^2}\right)\right\}, $$

which does not converge to the characteristic function of a normal r.v. In fact, for $|t|<\lambda$ (using the Taylor series for $\ln$),

$$ \varphi_{Y_n}(t)=\exp\left\{-\sum_{k=1}^\infty \frac{(-1)^{k+1}}{k}\frac{(t/\lambda)^{2k}}{(1-\rho^{2k})} \right\}, $$

which is approximately the c.f. of a normal r.v. for small values of $t$. So, in general, it's hard to find the stationary distribution of $Y$-s given Laplace innovations. The pdf of $Y_n$ ($h_Y$) can be found as the solution of

$$h_Y(x)=\int f_X(x-\rho u)h_Y(u)du$$

via the following recursion

$$h_{Y,n}(x)=\int f_X(x-\rho u)h_{Y,n-1}(u)du$$

starting with some arbitrary pdf $h_{Y,0}$. It can be shown that $h_{Y,n}\rightarrow h$ as $n\rightarrow\infty$.

However, it seems that there is a typo in the question. For example, in part (a) the authors ask to compute the autocorrelation $R_X(k,j)$ which does not make sense because $X$-s are i.i.d. So, probably their intention was to specify the stationary distribution of $Y$-s. Then you can find the corresponding distribution of innovations by noticing that

$$\varphi_{Y_n}(t)=\varphi_{Y_{n-1}}(\rho t)\varphi_{X_n}(t)$$

which yields


so that $X_n$ is a mixture of a degenerate r.v. ($\delta_0$) and a $\text{Laplace}(0,\lambda^{-1})$ r.v. with weights $\rho^2$ and $1-\rho^2$, respectively.

  • $\begingroup$ Thanks forr the insight. The thought of a typo crossed my mind before. But is it not usual in the theory of AR processes to specfiy the distribution of the innovations and work out the statistical properties of the process. $\endgroup$
    – Iconoclast
    Aug 3, 2015 at 13:04
  • $\begingroup$ @Iconoclast: The "reversed" problem is also popular. Anyway a problem form the introductory chapter should not be that difficult. $\endgroup$
    – user140541
    Aug 3, 2015 at 18:15

This is not an answer but maybe it might generate some ideas:

By a recursive argument one can see that $$ Y_n = \rho^k Y_{n-k} + \sum^k_{i=0}\rho^i X_{n-i}. $$ Since $Y_n$ was defined as stationary we have $|\rho|<1$ so taking $k\rightarrow \infty$ we get $$ Y_n = \sum^{\infty}_{i=0}\rho^i X_{n-i}. $$ Now $$ \mathbb P(Y_n> X_n + 1/2) = \mathbb P(\rho Y_{n-1}> 1/2) =\mathbb P\left( \rho\sum^{\infty}_{i=0}\rho^i X_{i}> 1/2\right) $$

Edit: On the use of CLT:

Note that if we define $Z_i = \rho^i X_i$ then $\text{Var}(Z_i) = \rho^{2i}\text{Var}(X_i) =2 \rho^{2i}$ and hence the sum of their variances is a geometric series$$ s_n^2 := \sum^n_{k=1}2 \rho^{2i} = 2\frac{1-\rho^{2(n+1)}}{1-\rho^2} $$

If $\{Z_i \}$ satisfies Lindeberg's condition then CLT applies and $$ \lim_{n\rightarrow \infty} \frac{\sum^{n}_{i=0}Z_{i}}{s_n} = \frac{\sqrt{1-\rho^2}}{\sqrt{2}}\sum^{\infty}_{i=0}\rho^i X_{i} $$ is a standard Normal rv.

  • $\begingroup$ Thank you for the ideas. I was thinking along similar lines. For the infinite sum, I was thinking of applying the generalized CLT (Kolomogrov version). Do you think it applies? It works for Paretian tail distributions which have infinte variances like the Laplac distrubtion of IID variables we have as the innovation terms $X_n$. $\endgroup$
    – Iconoclast
    Aug 2, 2015 at 16:10
  • $\begingroup$ @Iconoclast I dont think it would be necessary to apply the generalized CLT because $X_n$ does not have power tails. In fact it has much thinner tails so maybe one can apply the usual CLT, although I am not familiar with the case where the sum is a linear combination with different coefficients $\rho^i$. But you are probably right that CLT should be applied somehow. $\endgroup$
    – user126540
    Aug 2, 2015 at 16:38
  • $\begingroup$ But the problem is that the usual CLT only applies to finite variance RVs, whereas the Laplace distribution as infinite variance, that is why I was appealing to the generalized CLT. Any thoughts? different $p^i$'s can be accounted for by using a CLT for independent not necessarily identical RVs (e.g., Lyuponov CLT). Is there another CLT (other than Kolmogrov's) which has infinite variances in its condition? $\endgroup$
    – Iconoclast
    Aug 2, 2015 at 16:41
  • $\begingroup$ @Iconoclast Sure you are not confusing it with Cauchy distribution or something? Laplace has finite variance since it is simply a double-sided exponential distribution. $\endgroup$
    – user126540
    Aug 2, 2015 at 16:45
  • $\begingroup$ Okay my bad. But the question still remains as to how to apply the CLT, as it works for standardized sums (divided by the variance) or sums that are averages $\endgroup$
    – Iconoclast
    Aug 2, 2015 at 16:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.