# Help with understanding the notation?

I'm studying time series and in one of my resources I encountered the following:

The linear model described by $\text{(2.1)}$ above can be conveniently written in a more general notation by defining the column vectors $\boldsymbol{z}_t=(x_{t1},x_{t2},\ldots,x_{tq})'$ and $\boldsymbol{\beta}=(\beta_1,\beta_2,\ldots,\beta_q)'$, where $'$ denotes transpose, so $\text{(2.1)}$ can be written in the alternate form $$x_t=\boldsymbol{\beta'}\boldsymbol{z}_t+w_t.\tag{2.2}$$ where $w_t\sim \text{iid}\,\,\text{N}(0,\sigma_w^2)$. It is natural to consider estimating the unknown coefficient vector $\boldsymbol{\beta}$ by minimizing the error sum of squares $$Q=\sum_{t=1}^nw_t^2=\sum_{t=1}^n(x_t-\boldsymbol{\beta'}\boldsymbol{z}_t)^2,\tag{2.3}$$ with respect to $\beta_1,\beta_2,\ldots,\beta_q$. Minimizing $Q$ yields the ordinary least squares estimator of $\boldsymbol{\beta}$ . This minimization can be accomplished by differentiating $\text{(2.3)}$ with respect to the vector $\boldsymbol{\beta}$ or by using the properties of projections. In the notation above, this procedure gives the normal equations $$\left(\sum_{t=1}^n\boldsymbol{z}_t\boldsymbol{z}'_t\right)\widehat{\boldsymbol{\beta}}=\sum_{t=1}^n\boldsymbol{z}_tx_t.\tag{2.4}$$ The notation can be simplified by defining $Z\,=\,[\boldsymbol{z}_1|\boldsymbol{z}_2|\cdots|\boldsymbol{z}_n]'$ as the $n\times q$ matrix composed of the $n$ samples of the input variables, the observed $n\times 1$ vector $\boldsymbol{x}\,=\,(x_1,x_2,\ldots,x_n)'$ and the $n\times 1$ vector of errors.

The normal equations, $\text{(2.4)}$, can be written as $$(Z'Z)\widehat{\boldsymbol{\beta}}=Z'\boldsymbol{x}\tag{2.6}$$ and the solution $$\widehat{\boldsymbol{\beta}}=(Z'Z)^{-1}Z'\boldsymbol{x}\tag{2.7}$$ when the matrix $Z'Z$ is nonsingular. The minimized error sum of squares $\text{(2.3)}$, denoted $SSE$, can be written as \begin{align} SSE&=\sum_{t=1}^n(x_t-\widehat{\boldsymbol{\beta}}'\boldsymbol{z}_t)^2\\ &=(\boldsymbol{x}-Z\widehat{\boldsymbol{\beta}})'(\boldsymbol{x}-Z\widehat{\boldsymbol{\beta}})\\ &=\boldsymbol{x}'\boldsymbol{x}-\widehat{\boldsymbol{\beta}}'Z'\boldsymbol{x}\\ &=\boldsymbol{x}'\boldsymbol{x}-\boldsymbol{x}'Z(Z'Z)^{-1}Z'\boldsymbol{x},\\ \end{align}\tag{2.8} to give some useful versions for later reference. The ordinary least squares estimators are unbiased, i.e.e, $E(\widehat{\boldsymbol{\beta}})=\boldsymbol{\beta}$, and have the smallest variance within the class of linear unbiased estimators.

If the errors $w_t$ are normally distributed, $\widehat{\boldsymbol{\beta}}$ is also the maximum likelihood estimator for $\boldsymbol{\beta}$ and is normally distributed with $$\color{red}{\boxed{\displaystyle\color{black}{\text{cov}(\widehat{\boldsymbol{\beta}})=\sigma_w^2\left(\sum_{t=1}^n\boldsymbol{z}_t\boldsymbol{z}'_t\right)^{-1}=\sigma_w^2(Z'Z)^{-1}=\sigma_w^2C,}}}\tag{2.9}$$ where $$C=(Z'Z)^{-1}\tag{2.10}$$ is a convenient notation for later equations.

The part which I have marked is the one I don't understand?!...is that autocovariance or what? Can someone write the formula more explicitly or something?

Thnx =)

-
Do you know matrix algebra? Do you know the linear regression model? This is a result about simple regression and the quantity as written here either could or could not be related to autocovariances depending on what is inside $z_t$. – Learner Jan 22 '13 at 12:08
Yes I do know =) But for some reason I don't understand that notation... – jjepsuomi Jan 22 '13 at 12:14
Please take a look at the formula for the variance matrix here en.wikipedia.org/wiki/… (second formula). This is a general result. Maybe if you try to think about it in that simpler context, things could get clearer. – Learner Jan 22 '13 at 12:25

In the red box you have (the inverse of) a matrix. The reason is that the vector (I suppress the $t$) $z:=(z_1,\dots,z_n)$ and $z^t:=(z_1,\dots,z_n)^t$ are multiplied w.r.t. the Kronecker product. You can find its definition at Kronecker Product.

The idea is to look at $z$ as a $1\times n$ matrix, while $z^t$ is a $n\times 1$ matrix. By definition, their product is a

$$(1\cdot n)\times (n\cdot 1)= n\times n$$

matrix.

-
+1 And thank you again @Avitus :) I truly appreciate for your effort answering...I think it is very important thing and what I will keep in mind when helping others myself :) – jjepsuomi Aug 5 '13 at 9:40
@jjepsuomi thanks for the kind words: you are welcome and...keep going :-) – Avitus Aug 12 '13 at 8:26