3
$\begingroup$

I'm taking my second term of statistics, and I find myself obsessed with an unnecessary detail...again.

As follows:

$$Y_t=\rho Y_{t-1}+u_t$$

That is to say, we are working with a time series. We assume our present value depends on our previous value. $\rho$ is assumed to be $<1$, so effects of values close in time is greater than the effects of values in the distant past.

$u_t$ is assumed to be a purely random term.The $u_i$s are all assumed to be normally distributed, with zero mean and unit variance.

$Y_0$ (our starting value) is assumed to be 0.

My textbook then claims (out of nowhere)that the variance of this expression is equal to:

$$\frac {1}{1-\rho^2}$$

That's what my question is all about, I am trying to figure out how to reach this expression.

Below, I will describe my (so far unsuccessful) attempt at doing so.


Our first formula is

$$Y_t=\rho Y_{t-1}+u_t$$

This can be rewritten as:

$$Y_t=\sum {\rho^{n-t}u_t}$$

This way the $\rho$ is raised to the power of 0 at $u_n$, 1 at $u_{n-1}$, 2 at $u_{n-2}$ and so forth. $\rho$ will be raised to the power of n-1 at our starting value.

Using the formula for geometric sums, this can be rewritten as:

$$Y_t=\frac{u_t-\rho^n u_t}{1-\rho}$$

Assuming that n approaches infinity (knowing that $|\rho|<1$),we can rewrite this as:

$$Y_t=\frac{u_t}{1-\rho}$$

We take the variance of this expression:

$$V(Y_t)=\frac{\sigma^2}{(1-\rho)^2}$$

Finally, according to the text we have unit variance. I think this means that the variance of each $u_t$ should equal 1.

This would leave us with:

$$V(Y_t)=\frac{1}{(1-\rho)^2}$$

This still isn't equal to $\frac {1}{1-\rho^2}$!

Could someone please tell me what I'm doing wrong?

$\endgroup$

2 Answers 2

1
$\begingroup$

You are looking at an AR(1) time series model. By repeated substitution, you get

$$Y_t = \sum_{i=0}^{n-1} \rho^i u_{t-i} + \rho^n Y_{t-n}$$

The first term on the RHS is a geometric sum; the second goes to zero for $|\rho| < 1$ as $n \to \infty$. So take the limit, then take the variance to get the required result.

Note that you use the fact that $u_t$ is "purely random", i.e. the $u_t$ being independent for different $t$'s allows you to take the variance of the sum without any covariance terms popping out (that is, the property $\mathrm{Var}(X + Y) = \mathrm{Var}(X) + \mathrm{Var}(Y)$ applies).

To be explicit, your second-last line of computation should be

$$ \mathrm{Var} (Y_t) = \sum_{i=0}^{\infty} \rho^{2i} \mathrm{Var}(u_{t-i})$$

Edit: Here's the line-by-line:

$$ \begin{align}Y_t &= \rho Y_{t-1} + u_t\\ &= \rho (\rho Y_{t-2} + u_{t-1}) + u_t\\ &= \rho^2 Y_{t-2} + \rho u_{t-1} + u_t\\ & ... \\ &= \sum_{i=0}^{n-1} \rho^i u_{t-i} + \rho^n Y_{t-n} \\ &= \sum_{i=0}^{\infty} \rho^i u_{t-i}\\ \mathrm{Var}(Y_t) &= \mathrm{Var} \left (\sum_{i=0}^{\infty} \rho^i u_{t-i} \right)\\ &=\ \sum_{i=0}^{\infty} \mathrm{Var}\left( \rho^i u_{t-i} \right) \\ &=\ \sum_{i=0}^{\infty} \rho^{2i}\mathrm{Var}\left( u_{t-i} \right) \\ &=\ \sum_{i=0}^{\infty} \rho^{2i}\\ &= \frac{1}{1-\rho^2} \end{align} $$

If you're not happy with writing the last line of the expression for $Y_t$ (since you're dealing with infinity, you might not be convinced that "$\rho^{\infty}Y_{t-\infty}$" is zero, although a heuristic argument is that the decay from $\rho$ is geometric, which is faster than any growth that might possibly happen, which would be linear; in particular, $\rho^n Y_{t-n} \to 0$ almost surely), you'll need to refer to Batman's answer for a more rigorous proof.

For a more intuitive (but still informal) visualisation or argument for justifying writing $Y_t$ as an infinite sum (when $|\rho|<1$), think about what happens to the time series $Y_t = \rho Y_{t-1} + 1$ for when $|\rho|<1$.

$\endgroup$
4
  • $\begingroup$ I feel you're onto something here, I'm not quite how you got that first expression though. Rewrote question text to better reflect my thought process. $\endgroup$
    – Magnus
    Dec 26, 2015 at 18:06
  • $\begingroup$ I've edited my answer to make things more explicit. $\endgroup$
    – Ken Wei
    Dec 28, 2015 at 14:51
  • $\begingroup$ I assume that $\sum_{i=0}^{\infty} \rho^{2i}\mathrm{Var}\left( u_{t-i} \right) =\sum_{i=0}^{\infty} \rho^{2i}\\$ because we assume unit variance, but why does $\sum_{i=0}^{\infty} \rho^{2i} = \frac{1}{1-\rho^2}$? Sorry for being daft ^^ $\endgroup$
    – Magnus
    Dec 28, 2015 at 19:58
  • $\begingroup$ $\rho^{2i} = (\rho^2)^i$ . In other words, your geometric ratio is $\rho^2$. The answer follows from the formula for the sum of an infinite geometric series. $\endgroup$
    – Ken Wei
    Dec 28, 2015 at 20:06
1
$\begingroup$

Lets say $u_t \sim N(0, \sigma^2)$.

You have $E[Y_t] = E[\rho Y_{t-1} + u_t] = \rho E[Y_{t-1}] + E[u_t] = \rho E[Y_{t-1}] =0$ (by the initial condition $Y_0=0$).

You also have $E[Y_t^2] = E[(\rho Y_{t-1} + u_t)^2] = E[\rho^2 Y_{t-1}^2 + 2 \rho Y_{t-1} u_t + u_t^2] = \rho^2 E[Y_{t-1}^2] + 2 \rho E[Y_{t-1}]E[u_t] + E[u_t^2] = \rho^2 E[Y_{t-1}^2] + E[u_t^2] = \rho^2 E[Y_{t-1}^2] + \sigma^2$.

Solve these two recurrences (they are both linear recurrences, so use generating functions or guess),and then use $E[Y_t^2] - (E[Y_t])^2$.

$\endgroup$
4
  • $\begingroup$ I think the key to this is to assume that $E(Y_{t-1}^2)$ equals $\sigma^2$. In that case, we could move that term to the other side. Assuming we have equal variance, we could then write that $\sigma^2(1-\rho^2)=\sigma^2$ Dividing both sides with $1-\rho^2$ we'll get $\sigma^2=\frac{\sigma^2}{1-\rho^2}$ Assuming that $\sigma^2=1$ (because of unit variance) we'll end up with $\sigma^2=\frac{1}{1-\rho^2}$ $\endgroup$
    – Magnus
    Dec 27, 2015 at 15:54
  • $\begingroup$ That's kinda weird though, this example is meant to prove that a time series is stationary when $|p|<1$. In order to use this solution, we have to assume constant variance, which is one of the things we are trying to prove in the first place ^^ $\endgroup$
    – Magnus
    Dec 27, 2015 at 15:56
  • $\begingroup$ You can't assume $E[Y_{t-1}^2] = \sigma^2$ -- if $E[Y_{t-1}] = 0$, then this means that you're assuming that the variance is $\sigma^2$. $\endgroup$
    – Batman
    Dec 27, 2015 at 15:58
  • $\begingroup$ Would it be better if I wrote $\sigma^2_{t-1}$? I feel like I need to end up with something similar in order to arive at that expression... $\endgroup$
    – Magnus
    Dec 27, 2015 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.