3
$\begingroup$

Suppose you have a OLS linear regression with $\beta_0, \beta_1$, you data is {$x,y$}.

Now, suppose we swap the labels of your data. $x\rightarrow y, y\rightarrow x.$

You get new $\beta':$ $\beta_0', \beta_1'$.

Here is the problem. What can you say about $\beta/\beta'?$ (for example, can it be 10?)


My attempt:

Let's first focus on $\beta_1$.

In OLS $\beta_1=S_{XY}/S_{XX}$, $\beta_1' = S_{YX}/S_{YY}$.

Thus, $\beta_1/\beta_1' = S_{YY}/S_{XX}$.

But the answer is that this ratio should be between 0 and 1. I got stuck.

$S_{YY}/S_{XX} = \frac{\sum(Y_i-Y^{bar})^2}{\sum(X_i-X^{bar})^2}$

$\endgroup$
  • $\begingroup$ Why do you think your ratio, $\beta_1/\beta'_1$, is between $0$ and $1$? It is the ratio of the sample variances, which could be any positive real number (including $10$, as requested in your question). $\endgroup$ – Not_Dustin May 18 '17 at 3:17
  • $\begingroup$ @wrek Product of the regression coefficients lies between 0 and 1. $\endgroup$ – L.V.Rao May 18 '17 at 4:18
  • $\begingroup$ I dont understand, @L.V.Rao $\endgroup$ – wrek May 18 '17 at 5:12
  • $\begingroup$ @wrek we have the result that $r^2=\beta_{yx}\cdot \beta_{xy}$ $\endgroup$ – L.V.Rao May 18 '17 at 5:16
  • $\begingroup$ Thanks for your reply Rao, what can you say about the division? $\endgroup$ – wrek May 18 '17 at 5:37
1
$\begingroup$

The claim is wrong.

Regression coefficients have the following property:

$$r^2= \beta_{yx} \cdot \beta_{xy},$$ where $r$ is the correlation between x and y.

The product of the regression coefficients lies between 0 and 1 as $0\leq r^2\leq 1$.

Hence, it can be concluded that the ratio of the regression coefficients cannot lie between 0 and 1.

$\endgroup$
  • $\begingroup$ more specifically. The pearson coefficient is defined as $r=S_{XY}/(S_{XX} S_{YY}) \rightarrow r^2 = S^2_{XY}/(S_{XX}S_{XY})$ $\endgroup$ – wrek May 18 '17 at 19:31
  • $\begingroup$ thus, the product of these two $\beta s = r^2$, and you know that $r^2$ must between 0 and 1, thus, that's what it is. $\endgroup$ – wrek May 18 '17 at 19:31
  • $\begingroup$ So are we saying if we take $a$ and $b$ such that $0 \leq a \, b \leq 1$, then neither $a/b$ nor $b/a$ cannot lie between 0 and 1? Why? $\endgroup$ – Just_to_Answer May 19 '17 at 5:56
  • $\begingroup$ Or, are we saying, they are not required to lie between 0 and 1? $\endgroup$ – Just_to_Answer May 19 '17 at 6:05
  • $\begingroup$ @Just_to_Answer Good point. is not required to lie between 0 and 1 may be correct wording. In fact, if one regression coefficient is greater than 1, the other must be less than 1. $\endgroup$ – L.V.Rao May 19 '17 at 6:17
0
$\begingroup$

Code for an R experiment is below.

x <- rnorm(100, mean = 0, sd = 2)
y <- rnorm(100, mean = 0, sd = 1)
z <- rnorm(100, mean = 0, sd = 0.5)
coef(lm(y~x))[2]/coef(lm(x~y))[2] #slope when y is response/ slope when x is response
coef(lm(y~z))[2]/coef(lm(z~y))[2] #slope when y is response/ slope when z is response
coef(lm(z~x))[2]/coef(lm(x~z))[2] #slope when z is response/ slope when x is response
#  results 
#  0.3002797 
#  5.583115  
#  0.05378355 

Values for the ratios of $\hat \beta_1$ swapping response and predictor variables are: 0.3003, 5.5831 and 0.0538. So they don't have to fall in between 0 and 1, but they could.

Without further constraints/specifications, the best one can say is along the lines of the statement the OP found. OP states that the ratio will be equal to the ratio of the standard deviation of the response to the standard deviation of the predictor. It's close - instead it is the ratio of the sample variances, as noted in @Not_Dustin's comments.

One can see this from the formula $\hat \beta_1 = r \, \dfrac{S_y}{S_x}$.

So,
$$\frac{\hat \beta_1}{\hat \beta'_1} = \frac{r \, \dfrac{S_y}{S_x}}{r \, \dfrac{S_x}{S_y}} = \dfrac{S^2_y}{S^2_x}.$$

$\endgroup$
  • $\begingroup$ I think the ratio of the estimates equals the ratio of the sample variances, not sample standard deviations. I noted this in the first comment to the OP. $\endgroup$ – Not_Dustin May 19 '17 at 16:51
  • $\begingroup$ @Not_Dustin: Yes, absolutely - you are correct. Thanks! I will edit to make this correction. $\endgroup$ – Just_to_Answer May 19 '17 at 20:00
  • $\begingroup$ Who's Not_Justin? I'm Not_Dustin. :) $\endgroup$ – Not_Dustin May 19 '17 at 20:48
  • $\begingroup$ : ) The answer edit page doesnt offer selections of the handles unlike the comment section - so I had to go off of memory. $\endgroup$ – Just_to_Answer May 19 '17 at 20:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.