Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I was reading this article related to autoregressive processes of order $1$. According to wiki it is given by

$$ x_t = \phi{x_{t-1}} + \epsilon \\ |\phi| < 1 \\ x_t|x_1,\ldots,x_{t-1} \sim N(\phi{x_{t-1},1}) $$

I didn't get what this function $$\phi$$ is and how come the conditional distribution has mean equal to $$\phi{x_{t-1}}$$

Also I was referring to this book Gaussian Markov Random Fields Theory and Applications and they have assumed that the marginal distribution of $x_1$ is normal with mean zero and variance $$\frac1{1-\phi^2}$$ and modeled the joint density of $x$,

$$\pi(x)=\pi(x_1)\pi({x_2|x_1})\cdots\pi(x_n|x_{n-1})$$ to be gaussian distribution with precision matrix given by

$$Q = ( (1) (-\phi) \cdots\cdots)$$ $$( (-\phi) ({1 + \phi^2}) (-\phi)\cdots\cdots)$$ and so on I didn't get how this precision matrix was obtained. Any clarifications?

share|cite|improve this question

I didn't get what this function $\phi$ is

It's not a function, just a fixed number.

is and how come the conditional distribution has mean equal to $\phi{x_{t-1}}$

Because, assuming $\epsilon_t$ is a iid $N(0,1)$ (white gaussian noise) :

$E(x_t |x_{t-1} ) = E(\phi x_{t-1} + \epsilon_t |x_{t-1}) = \phi x_{t-1} + E(\epsilon_t) = \phi x_{t-1}$

Further, assuming $x_t$ is weakly stationary, so that $E(x_t^m)=E(x^m_{t-1})$, we have

$$E(x_t) = \phi E(x_{t-1}) + E(\epsilon_t) \Rightarrow E(x_t) (1-\phi) = 0 \Rightarrow E(x_t)=0$$

and squaring both sides:

$$E(x_t^2) = E(\phi x_{t-1}+ \epsilon_t)^2 = \phi^2 E( x^2_{t-1}) + E(\epsilon_t^2) $$

(the cross terms vanishes because $\epsilon_t$ is independent of $x_{t-1}$) and so the variance of $x_t$ is given by

$$E(x_t^2) (1 -\phi^2) = E(\epsilon_t^2) \Rightarrow E(x_t^2) = \frac{1}{1 -\phi^2} $$

Added: Here's a derivation for the joint distribution. Let's take $n=4$ , and $a = \phi$ (less typing). Then

$$p(x_1 x_2 x_3 x_4 ) = p(x_4 |x_3) p(x_3 |x_2) p(x_2 |x_1) p(x_1)$$

The first factor is $N(a \, x_3, 1)$ and the same for the rest, until the last which is $N(0,1/(1-a^2))$, hence

$$ p(x_1 x_2 x_3 x_4 ) = \frac{1}{\left(\sqrt{2 \pi}\right)^4 } \exp{\left[ -\frac{1}{2} \left( (x_4- a x_3)^2 + (x_3- a x_2)^2 + (x_2- a x_1)^2 + (1-a^2) \, x_1^2\right)\right]} $$

The term inside parentheses can be expanded as

$$x_4^2 - 2 a x_3 x_4 + (1+a^2) x_3 - 2 a x_3 x_2 + (1+a^2) x_2 - 2 a x_2 x_1 + x_1^2$$

or, in matricial form:

$$ \begin{pmatrix} x_1 & x_2 & x_3 & x_4 \\ \end{pmatrix} \begin{pmatrix} 1 & -a & 0 & 0 \\ -a & 1 +a^2 & -a & 0 \\ 0 & -a & 1 +a^2& -a \\ 1 & 0 & -a & 1 \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \\ \end{pmatrix} $$

So, finally $$p({\bf x}) = \frac{1}{\left(\sqrt{2 \pi}\right)^4} \exp{\left( -\frac{1}{2} {\bf x^T Q x } \right)}$$

which is the formula a joint 4-D gaussian, with zero mean and covariance $\Sigma = Q^{-1}$ And so we have found the precision matrix (inverse of the correlation matrix).

share|cite|improve this answer
What about the precision matrix. How was it derived? I am referring to this book… – user31820 Jun 25 '12 at 20:54
Its just a matter of writing the joint gaussian as a product of conditionals. You're supposed to know already about multidimensional gaussian variables, and what the precision matrix is. Doesn't the book explain it? – leonbloy Jun 25 '12 at 21:05
Yeah I got what the precision matrix and multidimensional gaussian variables are. But what I didn't get is how come that we have the value like phi and 1+square(phi) in the precision matrix for the variables or the nodes that are neighbors to each other. I got the idea of conditionally independence thing. I am just concerned how the value arose – user31820 Jun 25 '12 at 21:17
added, see if you get that – leonbloy Jun 25 '12 at 21:57
Thanks for the derivation. However, I have one more question. In the book, the distribution of x1 is given as N(0,1/(1-phi^2)). But you have taken it to be N(0,1). Also it can be seen when you derived the variance of xt. Also I didn't get how to derive the covariance matrix sigma. It is inverse of precision matrix definitely. But how is it derived in the book – user31820 Jun 26 '12 at 0:19

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.