1
$\begingroup$

I have two questions regarding logistic regression.

1) I understand that the results of a logistic regression model yield a table stating coefficients together with a p-statistic for each variable . This $p$-statistic states whether the variable at hand is important at predicting the outcome variable. For example, given the null hypothesis that the independent variable at hand is not important at predicting the outcome variable, then if $p< 0.05$ we conclude that the variable is important indeed. My question: why do we have $p$-statistics for logistic regression and not for linear regression? Wouldn't a coefficient close to $0$ be enough to conclude that the independent variable at hand is not an important predictor?

2) I'm having a bit of trouble understanding the relationship between the logistic regression equation $p = 1/(1+e^{-y})$ and its logit counterpart $\log(p/(1-p)) = MX + B$. I understand that these two equations are equivalent, correct? Is it safe to explain their relationship (and relevance to our model) by saying that we are most interested in the outcome of $p = 1/(1+e^{-y})$ given that this outcome gives us the probability that each observation produces the desired outcome, while the logit equation is only important to solve for coefficients?

$\endgroup$
2
$\begingroup$

1 Certainly you get p-values for regression coefficients ( $\beta$ ) in simple linear regression. They also test the hypothesis $\beta = 0$. This extends to multiple linear regression where there are multiple regression coefficients. For logistic regression, how accurate and reliable the p-values for the regression coefficients are is a different question.

2 In this case

$$ \log(p/(1-p))=MX+B \\ \Rightarrow p/(1-p)=e^{MX+B} \\ \Rightarrow p=e^{MX+B}/(1+e^{MX+B})\\ \Rightarrow p = 1/(1+e^{-(MX+B)}) $$ In other words, $y=MX+B$ in your notation.

$\endgroup$
  • $\begingroup$ What do you meant that for logistic regression the accuracy and reliability of the p-values differs than from linear regression? $\endgroup$ – CS Student Mar 22 at 16:27
  • $\begingroup$ And how do you go from 𝑝/(1−𝑝)=𝑒𝑀𝑋+𝐵⇒𝑝=𝑒𝑀𝑋+𝐵/(1+𝑒𝑀𝑋+𝐵) $\endgroup$ – CS Student Mar 22 at 16:33
  • $\begingroup$ @CSStudent In the normal case, the theory leading to p-values based on t and F distributions is exact. I don't believe any such exact theory exists for binary regression. $\endgroup$ – PM. Mar 22 at 16:33
  • 1
    $\begingroup$ @CSStudent As for your second comment "how do you go..." please try to have a go at the re-arrangement yourself. Hint multiply both sides by (1-p) and continue. $\endgroup$ – PM. Mar 22 at 16:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.