# Variance of random walk model?

I'm taking my second term of statistics, and I find myself obsessed with an unnecessary detail...again.

As follows:

$$Y_t=\rho Y_{t-1}+u_t$$

That is to say, we are working with a time series. We assume our present value depends on our previous value. $\rho$ is assumed to be $<1$, so effects of values close in time is greater than the effects of values in the distant past.

$u_t$ is assumed to be a purely random term.The $u_i$s are all assumed to be normally distributed, with zero mean and unit variance.

$Y_0$ (our starting value) is assumed to be 0.

My textbook then claims (out of nowhere)that the variance of this expression is equal to:

$$\frac {1}{1-\rho^2}$$

That's what my question is all about, I am trying to figure out how to reach this expression.

Below, I will describe my (so far unsuccessful) attempt at doing so.

Our first formula is

$$Y_t=\rho Y_{t-1}+u_t$$

This can be rewritten as:

$$Y_t=\sum {\rho^{n-t}u_t}$$

This way the $\rho$ is raised to the power of 0 at $u_n$, 1 at $u_{n-1}$, 2 at $u_{n-2}$ and so forth. $\rho$ will be raised to the power of n-1 at our starting value.

Using the formula for geometric sums, this can be rewritten as:

$$Y_t=\frac{u_t-\rho^n u_t}{1-\rho}$$

Assuming that n approaches infinity (knowing that $|\rho|<1$),we can rewrite this as:

$$Y_t=\frac{u_t}{1-\rho}$$

We take the variance of this expression:

$$V(Y_t)=\frac{\sigma^2}{(1-\rho)^2}$$

Finally, according to the text we have unit variance. I think this means that the variance of each $u_t$ should equal 1.

This would leave us with:

$$V(Y_t)=\frac{1}{(1-\rho)^2}$$

This still isn't equal to $\frac {1}{1-\rho^2}$!

Could someone please tell me what I'm doing wrong?

You are looking at an AR(1) time series model. By repeated substitution, you get

$$Y_t = \sum_{i=0}^{n-1} \rho^i u_{t-i} + \rho^n Y_{t-n}$$

The first term on the RHS is a geometric sum; the second goes to zero for $|\rho| < 1$ as $n \to \infty$. So take the limit, then take the variance to get the required result.

Note that you use the fact that $u_t$ is "purely random", i.e. the $u_t$ being independent for different $t$'s allows you to take the variance of the sum without any covariance terms popping out (that is, the property $\mathrm{Var}(X + Y) = \mathrm{Var}(X) + \mathrm{Var}(Y)$ applies).

To be explicit, your second-last line of computation should be

$$\mathrm{Var} (Y_t) = \sum_{i=0}^{\infty} \rho^{2i} \mathrm{Var}(u_{t-i})$$

Edit: Here's the line-by-line:

\begin{align}Y_t &= \rho Y_{t-1} + u_t\\ &= \rho (\rho Y_{t-2} + u_{t-1}) + u_t\\ &= \rho^2 Y_{t-2} + \rho u_{t-1} + u_t\\ & ... \\ &= \sum_{i=0}^{n-1} \rho^i u_{t-i} + \rho^n Y_{t-n} \\ &= \sum_{i=0}^{\infty} \rho^i u_{t-i}\\ \mathrm{Var}(Y_t) &= \mathrm{Var} \left (\sum_{i=0}^{\infty} \rho^i u_{t-i} \right)\\ &=\ \sum_{i=0}^{\infty} \mathrm{Var}\left( \rho^i u_{t-i} \right) \\ &=\ \sum_{i=0}^{\infty} \rho^{2i}\mathrm{Var}\left( u_{t-i} \right) \\ &=\ \sum_{i=0}^{\infty} \rho^{2i}\\ &= \frac{1}{1-\rho^2} \end{align}

If you're not happy with writing the last line of the expression for $Y_t$ (since you're dealing with infinity, you might not be convinced that "$\rho^{\infty}Y_{t-\infty}$" is zero, although a heuristic argument is that the decay from $\rho$ is geometric, which is faster than any growth that might possibly happen, which would be linear; in particular, $\rho^n Y_{t-n} \to 0$ almost surely), you'll need to refer to Batman's answer for a more rigorous proof.

For a more intuitive (but still informal) visualisation or argument for justifying writing $Y_t$ as an infinite sum (when $|\rho|<1$), think about what happens to the time series $Y_t = \rho Y_{t-1} + 1$ for when $|\rho|<1$.

• I feel you're onto something here, I'm not quite how you got that first expression though. Rewrote question text to better reflect my thought process. Dec 26, 2015 at 18:06
• I've edited my answer to make things more explicit. Dec 28, 2015 at 14:51
• I assume that $\sum_{i=0}^{\infty} \rho^{2i}\mathrm{Var}\left( u_{t-i} \right) =\sum_{i=0}^{\infty} \rho^{2i}\\$ because we assume unit variance, but why does $\sum_{i=0}^{\infty} \rho^{2i} = \frac{1}{1-\rho^2}$? Sorry for being daft ^^ Dec 28, 2015 at 19:58
• $\rho^{2i} = (\rho^2)^i$ . In other words, your geometric ratio is $\rho^2$. The answer follows from the formula for the sum of an infinite geometric series. Dec 28, 2015 at 20:06

Lets say $u_t \sim N(0, \sigma^2)$.

You have $E[Y_t] = E[\rho Y_{t-1} + u_t] = \rho E[Y_{t-1}] + E[u_t] = \rho E[Y_{t-1}] =0$ (by the initial condition $Y_0=0$).

You also have $E[Y_t^2] = E[(\rho Y_{t-1} + u_t)^2] = E[\rho^2 Y_{t-1}^2 + 2 \rho Y_{t-1} u_t + u_t^2] = \rho^2 E[Y_{t-1}^2] + 2 \rho E[Y_{t-1}]E[u_t] + E[u_t^2] = \rho^2 E[Y_{t-1}^2] + E[u_t^2] = \rho^2 E[Y_{t-1}^2] + \sigma^2$.

Solve these two recurrences (they are both linear recurrences, so use generating functions or guess),and then use $E[Y_t^2] - (E[Y_t])^2$.

• I think the key to this is to assume that $E(Y_{t-1}^2)$ equals $\sigma^2$. In that case, we could move that term to the other side. Assuming we have equal variance, we could then write that $\sigma^2(1-\rho^2)=\sigma^2$ Dividing both sides with $1-\rho^2$ we'll get $\sigma^2=\frac{\sigma^2}{1-\rho^2}$ Assuming that $\sigma^2=1$ (because of unit variance) we'll end up with $\sigma^2=\frac{1}{1-\rho^2}$ Dec 27, 2015 at 15:54
• That's kinda weird though, this example is meant to prove that a time series is stationary when $|p|<1$. In order to use this solution, we have to assume constant variance, which is one of the things we are trying to prove in the first place ^^ Dec 27, 2015 at 15:56
• You can't assume $E[Y_{t-1}^2] = \sigma^2$ -- if $E[Y_{t-1}] = 0$, then this means that you're assuming that the variance is $\sigma^2$. Dec 27, 2015 at 15:58
• Would it be better if I wrote $\sigma^2_{t-1}$? I feel like I need to end up with something similar in order to arive at that expression... Dec 27, 2015 at 16:14