Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

I would like to find out the formula for CandidateAbility used in the European PISA-test, which tests 9th grade pupil's abilities. Unfortunately the agency which publishes the results does not provide many mathematical facts. They say they use a logit-function to determine pupils' abilities in terms of percentage of correctly solved problems from a fixed problem set and the average problem difficulty for that set (never mind the definition of that). Googling for "logit" revealed the following formula:

$$\mathrm{CandidateAbility} = \log \left( \frac{x}{1-x} \right) + \mathrm{AverageDifficulty}$$

where $x$ denotes the fraction of correctly solved problems.

Assuming $\mathrm{AverageDifficulty}=0$ for now, this is centered around 0.5, i.e. a pupil solving half of the problems gets assigned ability zero. However, the PISA-agency says that they center the scale around 0.625, i.e. a pupil solving 62.5 percent of the problems gets assigned 0.

Now I can imagine many ways of modifying the above formula to achieve this. The first that come to my mind are:

$$\mathrm{CandidateAbility} = \log \left( \frac{x}{1-x} \right) + \mathrm{AverageDifficulty} - \log \left( \frac{0.625}{1-0.625} \right),$$ just shifting the outcome of the formula, and

$$\mathrm{CandidateAbility} = \log \left( \frac{x-c}{1-(x-c)} \right) + \mathrm{AverageDifficulty}$$

where c=0.625-0.5 (the difference between the new and the old center), modifying the input into the log-term.

My question is: Is there any modification of the formula, either one of the above or something entirely different, which is most natural from a statistician's point of view? Any suggestion would be welcome and could be used to counter-check against the data that is provided by the PISA-agency. Thanks!

share|improve this question
    
That description they provide is almost devoid of mathematical content, I fear. My guess is what they're trying to describe is some version/variant of a Rasch model. They may be trying to "invert" the model to recover a particular pupil's strength coefficient. –  cardinal Mar 28 '11 at 14:28
    
Sorry for not mentioning it: They do indeed use a sort of Rasch model. –  Peter Mar 28 '11 at 15:39

3 Answers 3

Logistic regressions in testing usually come from Item Response Theory:

http://en.wikipedia.org/wiki/Item_response_theory

For the logit scale, shifts (say from $a$ to $b$) are always of the form

$L(b) = L(a) + C$

where $L(x) = \log (x/(1-x))$ is the logit function.

If ability is centered at 0, has an additive effect on performance measured in the logit scale, and ability 0 corresponds to 62.5 percent correct solutions, then:

$L( \text{Percentage Solved}) = L(62.5) + \text{Ability}$

To get percentage or ability, use the inverse function of $L$, which is $e^L / (1+e^L)$.

share|improve this answer

Since the logit function needs inputs from the $(0,1)$ interval, your second formula is out of question. On the other hand, the first formula looks like a reasonable guess.

share|improve this answer

Is this the item you found on the web? I couldn't write as vaguely or as badly as that if I tried. The title says "What is a logit?". Any respectable answer to that question would have to say that the logit of a number $p$ between $0$ and $1$ is $\log(p/(1-p))$.

http://en.wikipedia.org/wiki/Logit

It looks to me as if they may be trying to say that "CandidateAbility" is defined as the logit of the probability that a candidate will get a correct answer plus an amount added to that so that CandidateAbility will be $0$ when that probability is $1/2$. But you have to read between the lines to guess that that may be what is meant. I am offended by their lack of clarity.

If they take that probability to be the actual proportion of right answers that a candidate got, then the CandidateAbility is $\infty$ if the candidate gets ever answer right.

They seem to have set up the definition so that "average ability" means getting 50% of the answers right. That seems horribly misleading.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.