# Autocovariance function of autoregressive stochastic process

I'm stuck on one of my exercises. I worked out an solution, which I think is correct, but differs from the given answer. Is my solution correct?

Suppose $\{Z_n\, n\in\mathbb{Z}\}$ is a sequence of uncorrelated r.v.'s, and $E\left[Z_n\right] = 0, Var\left[Z_n\right] = 1$. Define a autoregressive model

$$X_n = \alpha X_{n-1} + Z_n, \ |\alpha|<1$$ Prove that

$$Cov(X_n,X_{n+m}) = \frac{\alpha^{|m|}}{1-\alpha^2}, n, m\in \mathbb{Z}$$

My solution:

First suppose that $m > 0$. Observe that $$X_n = \sum_{i=0}^{n-1} \alpha^{i} Z_{n-i}$$ We have $$Cov(X_n,X_{n+m}) = \sum_{i=0}^{n-1}\sum_{j=0}^{n+m-1}\alpha^{i+j} Cov(Z_{n-i},Z_{n+m-j})$$

Note that $Cov(Z_{n-i}, Z_{n+m-j})=1$ if and only if $j = m+i$, i.e. given each $i = 0,\cdots n-1$, there corresponds only one $j = m+i$, which ranges between $m, n+m-1$, s.t. the term of the above sum does not vanish. Therefore,

$$Cov(X_n,X_{n+m}) = \sum_{i=0}^{n-1}\alpha^{2i+m} = \frac{\alpha^m(1-\alpha^{2n})}{1-\alpha^2}$$

Which definitely differs from the provided formula.

However, I noticed that the result I've got equates the provided formula, if $n \rightarrow \infty$. Therefore,

1. Is the exercise wrong, or missing some prerequisites?
2. Is my understanding of the exercise, especially of $X_n$, correct? For I in my solution supposed $X_1 = Z_1$.
3. Is my deduction wrong? What should the exercise be correctly solved?

P.S. I'm still working on stochastic process and am not familiar of autoregressive model, actually. All I know is from the definition given by the exercise.

• Can you provide a source for this problem? I actually think that there is a slight subtlety in the problem, one must assume that at least one of the variables $(X_n)_{n\in\mathbb Z}$ has a finite second moment, as this does not appear to follow directly from the autoregressive equations. – pre-kidney Mar 8 '17 at 23:22
• @pre-kidney it is from a stochastic textbook, written by Zhonggen Su, my teacher. But I'm not sure if there's an English version; I translated everything it has. Perhaps $Var[Z_n]=1$ ensured the existence of finite second moment of $X_n$'s? – R. Feng Mar 8 '17 at 23:54
• The missing assumption is that the joint process $(X_n,Z_n)_{n\in\mathbb Z}$ is weakly stationary. – pre-kidney Mar 11 '17 at 21:38

You have assumed that $X_0=0$, which violates the autoregressive equation at $n=0$. Instead, the system starts from 0 in the far distant past, and the correction term in your formula disappears.
More precisely, the formula for $X_n$ becomes $$X_n=\sum_{i=0}^{\infty}\alpha^i Z_{n-i},$$ and therefore for all $m\geq 0$ by the same argument as yours we obtain that $$\text{Cov}(X_n,X_{n+m})=\sum_{i=0}^{\infty}\alpha^{2i+m}=\frac{\alpha^m}{1-\alpha^2}.$$
• Can I understand it this way: we have infinitely many $Z_n$'s where n is negative integer, and if we start $X_n$ close enough to $-\infty$, the covariance can be arbitrarily close to the result? I seemed to miss that n could be negative and am confused how $X_n$ "starts" since n does not have a minimum. – R. Feng Mar 8 '17 at 16:51
• The process doesn't start at any finite $n$. Think of the autocovariance equations as a system of equations that (may or may not) have a solution. (It would be a separate exercise to show that this system of equations has a solution that is unique in distribution.) One way to construct a solution would be to use the infinite sum for $X_n$ in my post as the definition, given the $Z_n$ sequence. Also, if this is an exercise you may want to convince yourself that the infinite sum converges almost surely. – pre-kidney Mar 8 '17 at 18:42
• Thank you! Now I know the general idea and I'll figure out the uniqueness and a.s. convergence. My intuition tells that both holds because $|\alpha|<1$. Thanks again! – R. Feng Mar 8 '17 at 23:49