For an OLS model $y=X\beta +\epsilon$ it is well known that any of the estimated coefficients, say $\widehat\beta_i$, divided by its estimated standard error $s_{\widehat\beta_i}$, must follow a $t$-distribution whose degree of freedom equal is $n-p$ where $p$ is the number of explanatory variables (including intercept).

Now, suppose we linearly transform the model by a non-singular matrix $C$: $$ \begin{align} y &= X\beta + \epsilon\\ &=XC C^{-1}\beta + \epsilon\\ &= XC\color{red}{\gamma} + \epsilon \end{align} $$ Then in this new model where the coeffcients to find are $\gamma$, we can find an OLS estimator $\widehat\gamma$ which is again a BLUE. But the question is, is there any way we can tell about the distribution of $\widehat\gamma$'s $t$-statistic $\widehat\gamma_i/s_{\widehat\gamma_i}$? For example, I believe it is still $t$-distributed, but how can we prove it? And what will be its dof?


  • $\begingroup$ Are you cared about the asymptotic distribution? $\endgroup$ – RScrlli Nov 1 '18 at 17:14

The (exact) $t$ distribution arises not from the OLS itself but from the combination of OLS, multivariate normal vector $\epsilon$ with covariance matrix of $\sigma ^ 2I$, and by assuming that the null hypothesis of $\gamma_i = 0$ is true.

In such a case, the OLS vector of $\hat{\gamma}$ or $\hat{\beta}$ is a linear transformation of $y$, i.e., $\hat{\beta} = (X'X)^{-1}X'y$, hence it is multivariate by itself and the marginal distribution of each estimator $\hat{\beta}_i$ is $N(0, \sigma_{i}^2)$. The residuals vector $e$ is also a linear transformation of $y$, i.e., $(I - H)y$, hence it is also follows a multivariate distribution and then you can show that $\hat{\sigma}_i^2 \sim \sigma_i ^ 2\chi^2(n-p)$, hence the ratio $$ \hat{\beta}_i/\hat{\sigma}_i $$ by definition follows $t$ distribution with $n-p$ dof. The explicit derivations can be found on wikipedia or any standard book on linear regression.

Having said that, regressing on $XC$ and $\gamma = X^{-1}\beta$ induces no change to this properties. Namely, you transformed the data $X$ and accordingly the coefficients vector $\beta$, you induced no change to the assumptions on $\epsilon$ or the estimation method (OLS), hence same properties will follow from the same reasoning. The only difference is that the BLUE property is now w.r.t. $\gamma$ given $XC$ and not w.r.t. $\beta$ given $X$.

If no normality assumption on $\epsilon$ is made then the normal distribution of $\hat{\beta}_i$ is merely an approximation for any finite sample size $n$, hence I would not bother at all with student's $t$ and will stick to the normal approximation.

  • $\begingroup$ Thanks! But why does $\hat{\beta}_i/\hat{\sigma}_i$ by definition follow a $t$-distribution with dof=1? Actually I think we need to know the dependencies between $\hat{\beta}_i$ and $\hat{\sigma}_i$, and the dof should be $n-p$ where $p$ is the dof of the model, right? $\endgroup$ – Vim Nov 2 '18 at 0:44
  • $\begingroup$ 1) Sure, $n-p$ dof. Corrected. 2) $t(k) = Z/\sqrt{\chi^2(k)/k}$. Where $\hat{\beta}_i$ is normal and $\hat{\sigma}_i = \hat{\sigma}_{\epsilon}g(X)$, then the ratio is standard normal divided by square root of chi square etc. The independence stems from the fact that the residuals that construct $\hat{\sigma}^2_{\epsilon}$ are orthogonal to the columns space of $X$. $\endgroup$ – V. Vancak Nov 2 '18 at 9:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.