# Linear and logistic regression

Let a sample $$(x,y) \in \mathbb{R}^{2n}$$ be given, where $$y$$ only attains the values $$0$$ and $$1$$. We can try to model this data set by either linear regression $$y_i = \alpha_0 + \beta_0 x_i$$ with the coefficients determined by the method of least squares or by logistic regression $$\pi_i = \frac{\exp(\alpha_1 + \beta_1 x_i)}{1+\exp(\alpha_1 + \beta_1 x_i)},$$ where $$\pi_i$$ denotes the probability that $$y_i = 1$$ under the given value $$x_i$$ and the coefficients are determined by the Maximum-Likelihood method. My question is whether the following statement holds true.

Claim: If $$\beta_0 > 0$$ ($$\beta_0 < 0$$), then $$\beta_1 > 0$$ ($$\beta_1 < 0$$).

I figure this could be due to the sign of the correlation coefficient. However, I could not connect this to the logistic regression.

• For a single explanatory variable, some experimentation suggests this may be correct Jan 14, 2021 at 12:58

If $$\beta_0>0$$, i.e., $$\frac{\partial}{\partial x}E[Y|X=x] = \beta_0 > 0$$, is suggests that increase in $$x$$ will increase the probability that $$Y=1$$, since $$E[Y|X=x] = P(Y=1|X=x) = p$$. Now, the logistic model is equivalent to $$\ln\left( \frac{p}{1-p} \right) = \alpha_1 + \beta_1x.$$ The right hand side can be viewed as linear approximation of $$\ln(p/(1-p))$$. Given that the original linear model is OK, and $$\beta_0 > 0$$, it translates into $$\partial / \partial x \ln(p/(1-p)) = (p(1-p))^{-1}\beta_0 > 0$$ for the $$\ln$$ odds model, namely, $$\beta_1$$ must be positive as well (as the slope of the linear approximation).