2
$\begingroup$

i'm solving a problem that involve a linear model, and i'm trying to get the distribution of the least square estimator $\beta$.

i found in a book that:

$\widehat{\beta}\sim N_{p}(\beta, (X^{\prime}X)^{-}\sigma^{2})$. but how can i get the distributions to $\beta_{i}$?.

in my case, the matrix $(X^{\prime}X)^{-}$, i found was:

$$ \frac 1 \alpha \begin{bmatrix} \sum_{i=1}^N x_i^2 & -\sum_{i=1}^N x_i \\ & \\ -\sum_{i=1}^N x_i & N\\ \end{bmatrix} $$

where $\alpha= N \sum_{i=1}^N x_i^2- \left[ \sum_{i=1}^N x_i \right]^2$ and i'm looking for the distribution of $\beta_1, \beta_2$.

$\endgroup$
  • $\begingroup$ For your model you have $N$ observations, the intercept is $\beta_1$, and the slope is $\beta_2$, correct? Also your inverse matrix $(X'X)^{-1}$ should have a factor $$\frac1{N\sum x_i^2-(\sum x_i)^2}$$ in front of it. $\endgroup$ – grand_chat Sep 22 '18 at 1:19
  • $\begingroup$ yes, sorry you are right. i have edited. But i'm still confuse cuz $\beta$ is a vector and $(X^{\prime}X)^{-} \sigma^{2}$ is a matrix. so what is the variance of $\beta$'s. $\endgroup$ – JohanR Sep 22 '18 at 1:43
0
$\begingroup$

The result that you found $$ \hat\beta\sim N_p(\beta, (X'X)^{-1}\sigma^2)\tag{*} $$ allows you to read off the solution to your problem. If your model is $$Y_i = \beta_1 + \beta_2x_i+\varepsilon_i,\qquad i=1,\ldots,N, $$ then interpret the assertion (*) to mean that the joint distribution of $(\hat\beta_1,\hat\beta_2)$ is bivariate normal ($p=2$) with mean $(\beta_1,\beta_2)$ and covariance matrix $(X'X)^{-1}\sigma^2$. So the marginal distribution of $\hat\beta_1$ is normal with mean $\beta_1$ and variance equal to the $(1,1)$ entry of the covariance matrix, which is $$ \frac{\sigma^2}{N\sum x_i^2-(\sum x_i)^2}\sum x_i^2=\sigma^2\frac{\sum x_i^2}{N\sum x_i^2-(\sum x_i)^2}.\tag1 $$ Similarly the distribution of $\hat\beta_2$ is normal with mean $\beta_2$ and variance equal to the $(2,2)$ entry of the covariance matrix, which is $$ \frac{\sigma^2}{N\sum x_i^2-(\sum x_i)^2}N=\sigma^2\frac{N}{N\sum x_i^2-(\sum x_i)^2}.\tag2 $$ In (1) and (2), the denominator can be rewritten $N\sum(x_i-\bar x)^2$.

If your model is $$ Y_i = \theta x_i + \varepsilon_i,\qquad i=1,\ldots,N, $$ then the design matrix $X$ is a column $[x_1, x_2,\ldots,x_N]'$ so $X'X=\sum x_i^2$ and the result (*) says that the estimator $\hat\theta$ is normal with mean $\theta$ and variance $\displaystyle\frac{\sigma^2}{\sum x_i^2}.$

$\endgroup$
  • $\begingroup$ @JohanR Please see my edit. It is fine to define the $x_i=t_i^2/2$ as you have done, assuming the $t_i$'s are nonrandom. Your model is now $Y_i=\theta t_i+\varepsilon_i$ with no intercept. It has only one parameter. $\endgroup$ – grand_chat Sep 22 '18 at 3:11
  • $\begingroup$ thank you, for your help. mmm i have a a little doubt in my case the linear model is of the form Yi=t2i/2θ+εi and i took xi=t2i/2, and to write the model as a canonical linear model i took Θ′=[0,θ], so β0=0,β1=θ, but even in this case has sense to say that Var(θ)=Nσ2α?. but what about Var(β0)? $\endgroup$ – JohanR Sep 22 '18 at 14:16
0
$\begingroup$

$$ \widehat \beta_1 = \big[ 1, 0\big] \widehat\beta. $$ Therefore \begin{align} & \operatorname E\left(\widehat\beta_1\right) = \big[1,0\big]\operatorname E \left( \widehat\beta \right) = \big[1,0\big]\beta = \beta_1. \\[10pt] & \operatorname{var}\left(\widehat\beta_1\right) = \operatorname{var}\left( \big[1,0\big] \widehat\beta \right) = \big[1,0\big]\Big( \operatorname{var}\left(\widehat\beta\right) \Big) \left[ \begin{array}{c} 1 \\ 0 \end{array} \right] \\[10pt] = {} & \sigma^2 \big[1,0\big] \left( \frac 1 \alpha \begin{bmatrix} \sum_{i=1}^N x_i^2 & -\sum_{i=1}^N x_i \\ & \\ -\sum_{i=1}^N x_i & N\\ \end{bmatrix} \right) \left[ \begin{array}{c} 1 \\ 0 \end{array} \right] = \cdots \end{align} That gives you the expected value and the variance. And if $\widehat\beta$ is bivariate normal then a linear combination of its components with constant (i.e. non-random) coefficients is univariate normal.

And similarly for $\widehat\beta_2.$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.