I'm trying to work out the algebraic summations which yield the regression coefficients on the following linear model: $$ y_i = \alpha + \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_3 + \epsilon_i $$

If you center the variables, the answer in the two variable model is $$ \beta_1 = \frac{(\sum x_2^2)(\sum x_1y)-(\sum x_1x_2)(\sum x_2y)}{(\sum x_1^2)(\sum x_2^2)-(\sum x_1x_y)^2} \\ \beta_2 = \frac{(\sum x_1^2)(\sum x_2y)-(\sum x_1x_2)(\sum x_1y)}{(\sum x_1^2)(\sum x_2^2)-(\sum x_1x_y)^2} $$

I'm trying to work this out for the 3 variable case, but it's thwarting me. I'm aware that this can be simply expressed in matrix notation $(X'X)^{-1}X'Y$, but I'm trying to expand this for a piece of code optimization without having to make a call to the matrix operators.

  • $\begingroup$ Are you sure you need this optimization? Most modern matrix packages can do these even in double and still be competitive with scalar multiplication in terms of throughput. $\endgroup$ – Mortified Through Math May 6 '17 at 0:07
  • $\begingroup$ Sort of. The way the data is currently structured, it's optimized for particular types of scalar operations. Moving it out of that structure would require a much more significant change in code than simply working out this expansion. $\endgroup$ – WildGunman May 6 '17 at 0:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.