0
$\begingroup$

Let a sample $(x,y) \in \mathbb{R}^{2n}$ be given, where $y$ only attains the values $0$ and $1$. We can try to model this data set by either linear regression $$y_i = \alpha_0 + \beta_0 x_i$$ with the coefficients determined by the method of least squares or by logistic regression $$\pi_i = \frac{\exp(\alpha_1 + \beta_1 x_i)}{1+\exp(\alpha_1 + \beta_1 x_i)},$$ where $\pi_i$ denotes the probability that $y_i = 1$ under the given value $x_i$ and the coefficients are determined by the Maximum-Likelihood method. My question is whether the following statement holds true.

Claim: If $\beta_0 > 0$ ($\beta_0 < 0$), then $\beta_1 > 0$ ($\beta_1 < 0$).

I figure this could be due to the sign of the correlation coefficient. However, I could not connect this to the logistic regression.

$\endgroup$
1
  • $\begingroup$ For a single explanatory variable, some experimentation suggests this may be correct $\endgroup$
    – Henry
    Jan 14, 2021 at 12:58

1 Answer 1

1
$\begingroup$

If $\beta_0>0$, i.e., $\frac{\partial}{\partial x}E[Y|X=x] = \beta_0 > 0$, is suggests that increase in $x$ will increase the probability that $Y=1$, since $E[Y|X=x] = P(Y=1|X=x) = p$. Now, the logistic model is equivalent to $$ \ln\left( \frac{p}{1-p} \right) = \alpha_1 + \beta_1x. $$ The right hand side can be viewed as linear approximation of $\ln(p/(1-p))$. Given that the original linear model is OK, and $\beta_0 > 0 $, it translates into $\partial / \partial x \ln(p/(1-p)) = (p(1-p))^{-1}\beta_0 > 0$ for the $\ln$ odds model, namely, $\beta_1$ must be positive as well (as the slope of the linear approximation).

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .