I'm stuck on one of my exercises. I worked out an solution, which I think is correct, but differs from the given answer. Is my solution correct?

Suppose $\{Z_n\, n\in\mathbb{Z}\}$ is a sequence of uncorrelated r.v.'s, and $E\left[Z_n\right] = 0, Var\left[Z_n\right] = 1$. Define a autoregressive model

$$X_n = \alpha X_{n-1} + Z_n, \ |\alpha|<1$$ Prove that

$$Cov(X_n,X_{n+m}) = \frac{\alpha^{|m|}}{1-\alpha^2}, n, m\in \mathbb{Z}$$

My solution:

First suppose that $m > 0$. Observe that $$X_n = \sum_{i=0}^{n-1} \alpha^{i} Z_{n-i}$$ We have $$ Cov(X_n,X_{n+m}) = \sum_{i=0}^{n-1}\sum_{j=0}^{n+m-1}\alpha^{i+j} Cov(Z_{n-i},Z_{n+m-j}) $$

Note that $Cov(Z_{n-i}, Z_{n+m-j})=1$ if and only if $j = m+i$, i.e. given each $i = 0,\cdots n-1$, there corresponds only one $j = m+i$, which ranges between $m, n+m-1$, s.t. the term of the above sum does not vanish. Therefore,

$$ Cov(X_n,X_{n+m}) = \sum_{i=0}^{n-1}\alpha^{2i+m} = \frac{\alpha^m(1-\alpha^{2n})}{1-\alpha^2} $$

Which definitely differs from the provided formula.

However, I noticed that the result I've got equates the provided formula, if $n \rightarrow \infty$. Therefore,

  1. Is the exercise wrong, or missing some prerequisites?
  2. Is my understanding of the exercise, especially of $X_n$, correct? For I in my solution supposed $X_1 = Z_1$.
  3. Is my deduction wrong? What should the exercise be correctly solved?

P.S. I'm still working on stochastic process and am not familiar of autoregressive model, actually. All I know is from the definition given by the exercise.

  • $\begingroup$ Can you provide a source for this problem? I actually think that there is a slight subtlety in the problem, one must assume that at least one of the variables $(X_n)_{n\in\mathbb Z}$ has a finite second moment, as this does not appear to follow directly from the autoregressive equations. $\endgroup$ – pre-kidney Mar 8 '17 at 23:22
  • $\begingroup$ @pre-kidney it is from a stochastic textbook, written by Zhonggen Su, my teacher. But I'm not sure if there's an English version; I translated everything it has. Perhaps $Var[Z_n]=1$ ensured the existence of finite second moment of $X_n$'s? $\endgroup$ – R. Feng Mar 8 '17 at 23:54
  • $\begingroup$ The missing assumption is that the joint process $(X_n,Z_n)_{n\in\mathbb Z}$ is weakly stationary. $\endgroup$ – pre-kidney Mar 11 '17 at 21:38

You have assumed that $X_0=0$, which violates the autoregressive equation at $n=0$. Instead, the system starts from 0 in the far distant past, and the correction term in your formula disappears.

More precisely, the formula for $X_n$ becomes $$ X_n=\sum_{i=0}^{\infty}\alpha^i Z_{n-i}, $$ and therefore for all $m\geq 0$ by the same argument as yours we obtain that $$ \text{Cov}(X_n,X_{n+m})=\sum_{i=0}^{\infty}\alpha^{2i+m}=\frac{\alpha^m}{1-\alpha^2}. $$

  • $\begingroup$ Can I understand it this way: we have infinitely many $Z_n$'s where n is negative integer, and if we start $X_n$ close enough to $-\infty$, the covariance can be arbitrarily close to the result? I seemed to miss that n could be negative and am confused how $X_n$ "starts" since n does not have a minimum. $\endgroup$ – R. Feng Mar 8 '17 at 16:51
  • $\begingroup$ The process doesn't start at any finite $n$. Think of the autocovariance equations as a system of equations that (may or may not) have a solution. (It would be a separate exercise to show that this system of equations has a solution that is unique in distribution.) One way to construct a solution would be to use the infinite sum for $X_n$ in my post as the definition, given the $Z_n$ sequence. Also, if this is an exercise you may want to convince yourself that the infinite sum converges almost surely. $\endgroup$ – pre-kidney Mar 8 '17 at 18:42
  • $\begingroup$ Thank you! Now I know the general idea and I'll figure out the uniqueness and a.s. convergence. My intuition tells that both holds because $|\alpha|<1$. Thanks again! $\endgroup$ – R. Feng Mar 8 '17 at 23:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.