1
$\begingroup$

Let's say I have an AR(p) process $$ Y_t = a_0 + \sum_{i=1}^p a_p Y_{t-i} + \sigma\epsilon_t $$ with $\epsilon_t \sim \mathcal{N}(0,1)$, i.i.d., for $t = p+1,\ldots,N$. I can estimate the parameters $a_0,\ldots,a_p$ by finding the least-squares solution to $$ \underbrace{\begin{pmatrix} Y_{p+1}\\ Y_{p+2}\\ \vdots\\ Y_N \end{pmatrix}}_{Y} = \underbrace{\begin{pmatrix} 1 & Y_{p} & Y_{p-1} & \cdots & Y_1 \\ 1 & Y_{p+1} & Y_{p} & \cdots & Y_2 \\ \vdots & \vdots & \ddots & \vdots \\ 1 & Y_{N-1} & Y_{N-2} & \cdots & Y_{N-p} \end{pmatrix}}_{X} \begin{pmatrix} a_0\\ a_1\\ a_2\\ \vdots\\ a_p \end{pmatrix} $$ Given my least-squares solution for the coefficients, $\hat{a} := (\hat{a}_0,\ldots, \hat{a}_p) = (X^TX)^{-1}X^TY$, I can estimate the standard error of the coefficients by first computing the sample variance of the residuals $$ \hat{\sigma}^2 := \frac{1}{(N-p)-(p+1)}\sum_{t=p+1}^N \left(r_t - \bar{r}\right)^2 \qquad (1) $$ where $$ r_t := Y_t - \left(\hat{a}_0 + \sum_{i=1}^p \hat{a}_i Y_{t-i}\right), \qquad t = p+1,\ldots,N $$

Then, noting \begin{align*} Var(\hat{a}) & = Var((X^TX)^{-1}X^TY) \\ & = (X^TX)^{-1}X^TVar(Y)X(X^TX)^{-1} \\ & = Var(Y)(X^TX)^{-1}, \end{align*} my estimate for the standard error of the coefficient estimates is $$ \text{std err} := \hat{\sigma}\sqrt{(X^TX)^{-1}} $$

I have a couple of questions about this method:

  1. The residuals are clearly correlated. E.g. for $p = 3$, the first two residuals look like $$ \begin{pmatrix} r_4 \\ r_5 \end{pmatrix} = \begin{pmatrix} \color{red}{Y_4} - \hat{a}_0 - \hat{a}_1 \color{green}{Y_3} - \hat{a}_2\color{blue}{Y_2} - \hat{a}_3Y_1 \\ Y_5 - \hat{a}_0 - \hat{a}_1 \color{red}{Y_4} - \hat{a}_2\color{green}{Y_3} - \hat{a}_3\color{blue}{Y_2} \end{pmatrix} $$ thus the estimator (1) seems dubious. Is there an alternative? Or is (1) the standard in practice? If so, is there any correction for using dependent random variables to estimate sample variance?
  2. I subtracted $(p+1)$ from the total number of observations $(N-p)$ to account for the number of random variables used in the computation of (1): one dof from $\bar{r}$ and $p$ dof from the lagged $Y_t$ terms. Is this correct?
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.