# Confusion regarding autoregressive process

I was reading this article related to autoregressive processes of order $1$. According to wiki it is given by

$$x_t = \phi{x_{t-1}} + \epsilon \\ |\phi| < 1 \\ x_t|x_1,\ldots,x_{t-1} \sim N(\phi{x_{t-1},1})$$

I didn't get what this function $$\phi$$ is and how come the conditional distribution has mean equal to $$\phi{x_{t-1}}$$

Also I was referring to this book Gaussian Markov Random Fields Theory and Applications and they have assumed that the marginal distribution of $x_1$ is normal with mean zero and variance $$\frac1{1-\phi^2}$$ and modeled the joint density of $x$,

$$\pi(x)=\pi(x_1)\pi({x_2|x_1})\cdots\pi(x_n|x_{n-1})$$ to be gaussian distribution with precision matrix given by

$$Q = ( (1) (-\phi) \cdots\cdots)$$ $$( (-\phi) ({1 + \phi^2}) (-\phi)\cdots\cdots)$$ and so on I didn't get how this precision matrix was obtained. Any clarifications?

-

I didn't get what this function $\phi$ is

It's not a function, just a fixed number.

is and how come the conditional distribution has mean equal to $\phi{x_{t-1}}$

Because, assuming $\epsilon_t$ is a iid $N(0,1)$ (white gaussian noise) :

$E(x_t |x_{t-1} ) = E(\phi x_{t-1} + \epsilon_t |x_{t-1}) = \phi x_{t-1} + E(\epsilon_t) = \phi x_{t-1}$

Further, assuming $x_t$ is weakly stationary, so that $E(x_t^m)=E(x^m_{t-1})$, we have

$$E(x_t) = \phi E(x_{t-1}) + E(\epsilon_t) \Rightarrow E(x_t) (1-\phi) = 0 \Rightarrow E(x_t)=0$$

and squaring both sides:

$$E(x_t^2) = E(\phi x_{t-1}+ \epsilon_t)^2 = \phi^2 E( x^2_{t-1}) + E(\epsilon_t^2)$$

(the cross terms vanishes because $\epsilon_t$ is independent of $x_{t-1}$) and so the variance of $x_t$ is given by

$$E(x_t^2) (1 -\phi^2) = E(\epsilon_t^2) \Rightarrow E(x_t^2) = \frac{1}{1 -\phi^2}$$

Added: Here's a derivation for the joint distribution. Let's take $n=4$ , and $a = \phi$ (less typing). Then

$$p(x_1 x_2 x_3 x_4 ) = p(x_4 |x_3) p(x_3 |x_2) p(x_2 |x_1) p(x_1)$$

The first factor is $N(a \, x_3, 1)$ and the same for the rest, until the last which is $N(0,1/(1-a^2))$, hence

$$p(x_1 x_2 x_3 x_4 ) = \frac{1}{\left(\sqrt{2 \pi}\right)^4 } \exp{\left[ -\frac{1}{2} \left( (x_4- a x_3)^2 + (x_3- a x_2)^2 + (x_2- a x_1)^2 + (1-a^2) \, x_1^2\right)\right]}$$

The term inside parentheses can be expanded as

$$x_4^2 - 2 a x_3 x_4 + (1+a^2) x_3 - 2 a x_3 x_2 + (1+a^2) x_2 - 2 a x_2 x_1 + x_1^2$$

or, in matricial form:

$$\begin{pmatrix} x_1 & x_2 & x_3 & x_4 \\ \end{pmatrix} \begin{pmatrix} 1 & -a & 0 & 0 \\ -a & 1 +a^2 & -a & 0 \\ 0 & -a & 1 +a^2& -a \\ 1 & 0 & -a & 1 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ \end{pmatrix}$$

So, finally $$p({\bf x}) = \frac{1}{\left(\sqrt{2 \pi}\right)^4} \exp{\left( -\frac{1}{2} {\bf x^T Q x } \right)}$$

which is the formula a joint 4-D gaussian, with zero mean and covariance $\Sigma = Q^{-1}$ And so we have found the precision matrix (inverse of the correlation matrix).

-
What about the precision matrix. How was it derived? I am referring to this book books.google.com/… – user31820 Jun 25 '12 at 20:54
Its just a matter of writing the joint gaussian as a product of conditionals. You're supposed to know already about multidimensional gaussian variables, and what the precision matrix is. Doesn't the book explain it? – leonbloy Jun 25 '12 at 21:05
Yeah I got what the precision matrix and multidimensional gaussian variables are. But what I didn't get is how come that we have the value like phi and 1+square(phi) in the precision matrix for the variables or the nodes that are neighbors to each other. I got the idea of conditionally independence thing. I am just concerned how the value arose – user31820 Jun 25 '12 at 21:17
added, see if you get that – leonbloy Jun 25 '12 at 21:57
Thanks for the derivation. However, I have one more question. In the book, the distribution of x1 is given as N(0,1/(1-phi^2)). But you have taken it to be N(0,1). Also it can be seen when you derived the variance of xt. Also I didn't get how to derive the covariance matrix sigma. It is inverse of precision matrix definitely. But how is it derived in the book – user31820 Jun 26 '12 at 0:19