15
$\begingroup$

What is the meaning of 'Sxx' and 'Sxy' in simple linear regression? I know the formula but what is the meaning of those formulas?

$\endgroup$
1
  • 1
    $\begingroup$ Possibly $\sum x^2$ and $\sum xy$, or $\sum (x-\bar{x})^2$ and $\sum (x-\bar{x})(y-\bar{y})$ but it would depend on how the writer used them $\endgroup$
    – Henry
    Oct 27, 2015 at 7:51

2 Answers 2

18
$\begingroup$

$S_{xx}$ is the sum of the squares of the difference between each $x$ and the mean $x$ value.

$S_{xy}$ is sum of the product of the difference between $x$ its means and the difference between $y$ and its mean.

So $S_{xx}=\Sigma(x-\overline{x})(x-\overline{x})$ and $S_{xy}=\Sigma(x-\overline{x})(y-\overline{y})$. Both of these are often rearranged into equivalent (different) forms when shown in textbooks.

$\endgroup$
3
  • 1
    $\begingroup$ It depends. Look at en.wikipedia.org/wiki/Simple_linear_regression to see what Henry wrote. $\endgroup$ Oct 27, 2015 at 7:54
  • $\begingroup$ Fair enough. Given that the OP knows the formulas but couldn't see a direct link between name and formula I assumed it wasn't the other meaning. $\endgroup$
    – Ian Miller
    Oct 27, 2015 at 8:00
  • $\begingroup$ At the bottom of that wikipedia articel, it uses simply $S_{xx} = \sum{x_i}$. Is that true? $\endgroup$
    – user71207
    Sep 22, 2021 at 2:31
10
$\begingroup$

To add to the answer from Ian Miller,

$$S_{xx}=\sum x^2 -\frac{(\sum x)^2}{n}=\sum x^2 -n\bar{x}^2$$

Intuitively, $S_{xy}$ is the result when you replace one of the $x$'s with a $y$.

$$S_{xy}=\sum xy -\frac{\sum x \sum y}{n}=\sum xy -n\bar{x}\bar{y}$$

Also, just for your information, the good thing about this notation is that it simplifies other parts of linear regression.

For example, the product-moment correlation coefficient:

$$r=\frac{\sum xy -n\bar{x}\bar{y}}{\sqrt{(\sum x^2 -n\bar{x}^2)(\sum y^2 - n\bar{y}^2)}} = \frac{S_{xy}}{\sqrt{S_{xx}S_{yy}}}$$

or to find the gradient of the best-fit line $y=a+bx$:

$$y-\bar{y}=b(x-\bar{x}), \text{ where } b=\frac{S_{xy}}{S_{xx}}$$

The pragmatic importance of this is that if you are doing a long question about linear regression, calculating $S_{xx}$, $S_{yy}$ and $S_{xy}$ at the beginning can save you a lot of work.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.