1
$\begingroup$

Suppose that $\hat\beta$ is the OLS estimator of the multiple linear regression given by $$ \hat\beta =\underset{\beta}{\operatorname{arg\,min}}\Bigl\{\sum_{i=1}^n(y_i-\beta_0-\sum_{j=1}^px_{ij}\beta_j)^2\Bigr\} $$ and $\hat\beta^c$ is the OLS estimator of the multiple linear regression when the variables are centered, i.e. $$ \hat\beta^c =\underset{\beta^c}{\operatorname{arg\,min}}\Bigl\{\sum_{i=1}^n((y_i-\bar y)-\beta_0^c-\sum_{j=1}^p(x_{ij}-\bar x_j)\beta_j^c)^2\Bigr\}. $$ It seems that $\hat\beta_0^c$ should be equal to $0$. We have that \begin{align*} \hat\beta &=\underset{\beta}{\operatorname{arg\,min}}\Bigl\{\sum_{i=1}^n(y_i-\beta_0-\sum_{j=1}^px_{ij}\beta_j)^2\Bigr\}\\ &=\underset{\beta}{\operatorname{arg\,min}}\Bigl\{\sum_{i=1}^n((y_i-\bar y+\bar y)-\beta_0-\sum_{j=1}^p(x_{ij}-\bar x_j+\bar x_j)\beta_j)^2\Bigr\}\\ &=\underset{\beta}{\operatorname{arg\,min}}\Bigl\{\sum_{i=1}^n((y_i-\bar y)+\underbrace{\bar y-\beta_0-\sum_{j=1}^p\bar x_j\beta_j}_{=\beta_0^c}-\sum_{j=1}^p(x_{ij}-\bar x_j)\underbrace{\beta_j}_{=\beta_j^c})^2\Bigr\}. \end{align*}

It is straightforward to see that $\hat\beta_0^c=0$ in the simple linear regression. We have that $$ \bar y-\hat\beta_0-\bar x\hat\beta_1=\bar y-\bar y+\bar x\hat\beta_1-\bar x\hat\beta_1=0. $$

How can we show that this also holds for the multiple linear regression, i.e. how can we show that $\hat\beta_0^c$ is equal to $0$?

Any help is much appreciated!

$\endgroup$
1
$\begingroup$

Consider the partitioned regression: $y_i=X_{i1}^\top\beta_1+X_{i2}^\top\beta_2+\epsilon_i$. It is easy to show that $$ \hat{\beta}_1=(\mathbf{X}_1^\top\mathbf{X}_1)^{-1}\mathbf{X}_1^{\top}(\mathbf{y}-\mathbf{X}_2\hat{\beta}_2), $$ where $\mathbf{y}\equiv[y_1,\ldots,y_n]^\top$ and $\mathbf{X}_j\equiv[X_{ij},\ldots,X_{nj}]^{\top}$. If $X_{i1}=1$, the last equation reduces to $$ \hat{\beta}_1=\mathbf{i}^\top(\mathbf{y}-\mathbf{X}_2\hat{\beta}_2)/n. $$ When $y_i$ and $X_{i2}$ are measured as deviations from the corresponding sample means, $\mathbf{i}^\top\mathbf{y}=0$ and $\mathbf{i}^\top\mathbf{X}_2=\mathbf{0}$, which yields $\hat{\beta}_1=0$.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

Consider the following multiple regression: $$y_t^*=\beta_0+\beta_1x_{1t}^*+...+\beta_nx_{nt}^*+\epsilon_t,$$ where $y_t^*=y_t-\bar{y}; x_{it}^*=x_{it}-\bar{x_i}$ for $i=1,...,n$. By applting OLS we have: $$\sum_{t=1}^k(y_t^*-\beta_0-\beta_1x_{1t}^*-...-\beta_nx_{nt}^*)^2 \rightarrow min.$$ Since we are interested in determining $\beta_0$, we have $$\beta_0=\frac{1}{n}\sum_{t=1}^k(y_t^*-\beta_1x_{1t}^*-...-\beta_nx_{nt}^*)=\frac{1}{n}\sum_{t=1}^{k}(y_{t}-\bar{y})-\frac{\beta_1}{n}\sum_{t=1}^{k}(x_{1t}-\bar{x_1})-...\frac{\beta_n}{n}\sum_{t=1}^{k}(x_{nt}-\bar{x_n})=0$$ The above equality holds since $\frac{1}{n}\sum_{t=1}^{k}(y_{t}-\bar{y})=\bar{y}-\bar{y}=0;\frac{\beta_i}{n}\sum_{t=1}^{k}(x_{it}-\bar{x_i})=\beta_i(\bar{x_i}-\bar{x_i})=0$ for all $i=1,...,n$.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.