Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

A logistic regression is meant for a binary/categorical variable. Sort of like age vs baldness.

1) So, does the "S-curve" regression equation output give the odds of having that condition for a given x-value (eg: age), since the values go from 0 to 1 on the Y-axis?

(thinking to myself....)If the data models this behavior strongly (age vs. ability to vote), then it will be a very sharp cutoff at 18, and I guess it will be pretty accurate, yielding 0% and 100% for almost all ages... With more ambiguity, I guess the curve will not go from 0 to 1 on the y-axis, but more like 25% to 75%, for example?

2) Anyone have a good example of a binary data set that exhibits this binary nature?

3) How does one do Logistic Regression in Excel 2010?
I only see Layout -> Trendline -> Logarithmic

share|improve this question
add comment

2 Answers 2

$\newcommand{\logit}{\operatorname{logit}}$

The popular confusion between probability and odds seems to be in play here. A probability $p$ is always in the interval $[0,1]$. The odds in favor of an event or a statement is the number $p/(1-p)$, where $p$ is the probability. The odds is in the interval $[0,\infty]$ (closed brackets at both ends), and is more than $1$ when the probability is more than $1/2$. What is commonly called "$3$-to-$1$ odds" would mean $p/(1-p)=3$, so that $p=3/4$. The probability is $3/4$; the odds is $3$.

In logistic regression one has a real-valued predictor variable $x$ observed in $n$ cases, thus a vector $(x_1,\ldots,x_n)$, and a $\{0,1\}$-valued response variable $y$ observed in the same $n$ cases, $(y_1,\ldots,y_n)$. One estimates a function $$ \logit p(x) = ax+b, $$ where $\logit p = \log\dfrac{p}{1-p}$, so that $p$ must be between $0$ and $1$. The function $p(x)$ is supposed to be an estimated probability that $y=1$ given the value of $x$. The values of $a$ and $b$ determine the function $p$, and they are estimated based on the observed values $(x_1,\ldots,x_n)$ and $(y_1,\ldots,y_n)$, using maximum likelihood. The likelihood function is $$ L(a,b) = \left(\prod_{i\ :\ y_i=1} p(x_i)\right)\left(\prod_{i\ :\ y_i=0} (1-p(x_i)). \right) $$ The values $\hat a$ and $\hat b$ of $a$ and $b$ that maximize this are the estimates. They are found by iterative numerical methods.

If all $y$ values corresponding to $x<\text{cutoff}$ are $0$ and all $y$ values corresponding to $x>\text{cutoff}$ are $1$, then one does get $p(x)=1\text{ or }0$ according as $x$ is larger or smaller than the cutoff, and that means $a=+\infty$. and for things like age and baldness (if baldness can really be considered binary) one would get a finite number for $a$, which would be positive if baldness is more frequent for older people in the observed dataset. And $a$ would be $0$ if baldness is uncorrelated with age, and negative if baldness is more frequent among younger people.

But, except in the trivial case where $a=0$, so $p$ is constant, the function $p$ will always satisfy either $$ p(x)\to1\text{ as }x\to+\infty\text{ and }p(x)\to0\text{ as }x\to-\infty $$ or

$$ p(x)\to1\text{ as }x\to-\infty\text{ and }p(x)\to0\text{ as }x\to+\infty. $$

share|improve this answer
add comment

Michael Hardy gave you a great answer, but I wanted to follow up a bit, especially on parts 2 and 3 of your question.

The short answer to part 1 is that the S-curve gives the probability, not the odds.

Logistic regression is not really intended to model situations with a discontinuity like what you have described with "ability to vote" and "age" where the function jumps from 0 to 1 at age 18. However, if you modeled voting behavior (that is, whether or not a person actually votes), then it would probably work quite well as the probability that a person actually votes is not 100%.

For part 2 of your question, here is a completely described logistic regression example. This is a marketing example that involves choosing what ads to show a customer in order to increase click through rate.

For part 3, there is not an easy way to do logistic regression in Excel (at least that I know about). It is not a part of the standard Data Analysis add in. You can, however, program the optimization necessary to maximize the likelihood function shown in Michael Hardy's post to find the maximum likelihood estimates using Solver. But this is not that easy. Much easier to use a standard statistics package.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.