0
$\begingroup$

Let a sample $(x,y) \in \mathbb{R}^{2n}$ be given, where $y$ only attains the values $0$ and $1$. We can try to model this data set by either linear regression $$y_i = \alpha_0 + \beta_0 x_i$$ with the coefficients determined by the method of least squares or by logistic regression $$\pi_i = \frac{\exp(\alpha_1 + \beta_1 x_i)}{1+\exp(\alpha_1 + \beta_1 x_i)},$$ where $\pi_i$ denotes the probability that $y_i = 1$ under the given value $x_i$ and the coefficients are determined by the Maximum-Likelihood method. My question is whether the following statement holds true.

Claim: If $\beta_0 > 0$ ($\beta_0 < 0$), then $\beta_1 > 0$ ($\beta_1 < 0$).

I figure this could be due to the sign of the correlation coefficient. However, I could not connect this to the logistic regression.

$\endgroup$
1
  • $\begingroup$ For a single explanatory variable, some experimentation suggests this may be correct $\endgroup$
    – Henry
    Jan 14 at 12:58
0
$\begingroup$

If $\beta_0>0$, i.e., $\frac{\partial}{\partial x}E[Y|X=x] = \beta_0 > 0$, is suggests that increase in $x$ will increase the probability that $Y=1$, since $E[Y|X=x] = P(Y=1|X=x) = p$. Now, the logistic model is equivalent to $$ \ln\left( \frac{p}{1-p} \right) = \alpha_1 + \beta_1x. $$ The right hand side can be viewed as linear approximation of $\ln(p/(1-p))$. Given that the original linear model is OK, and $\beta_0 > 0 $, it translates into $\partial / \partial x \ln(p/(1-p)) = (p(1-p))^{-1}\beta_0 > 0$ for the $\ln$ odds model, namely, $\beta_1$ must be positive as well (as the slope of the linear approximation).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.