The probability of a student with a skill parameter of "s" to obtain at least a score of "k" in a certain test is defined as: $$\frac{1}{e^{b_k-s}+1}$$ Where $b_k$ is a difficulty parameter of achieving at least that score ($b_0$ is $-\infty$ since the student always gets at least 0 points). The maximum score in the test is m. In the data there are several tests (each with their own difficulty parameters, all with the same value of m), and several students (each with their own skill parameter).

For example, if the student achieves a score of 2 out of 5, that implies that they got at least 2 points but failed to get at least 3 points, so the likelihood is: $$\frac{1}{e^{b_2-s}+1}-\frac{1}{e^{b_3-s}+1}$$ (Getting at least 3 points implies getting more than 2 points, so the likelihood of achieving 2 points is the difference in likelihood between getting at least 2 points and at least 3 points)

But using that likelihood function for maximum likelihood estimation gives odd results for the difficulty parameters with fixed student skill parameters (a higher score doesn't always imply an equal or higher difficulty, or sometimes some difficulty parameters are undetermined), instead, using a likelihood function of: $$\frac{\left(1-\frac{1}{e^{b_3-s}+1}\right) \left(1-\frac{1}{e^{b_4-s}+1}\right) \left(1-\frac{1}{e^{b_5-s}+1}\right)}{\left(e^{b_1-s}+1\right) \left(e^{b_2-s}+1\right)}$$ (This treats a single score as 5 separate events, achieving at least 1 and 2 points, but failing to achieve at least 3, 4 and 5 points) Seems to work better for difficulty parameter estimation.

On the other hand, when estimating the skill parameters of the students, the first likelihood function seems to work better compared to the second one (the values obtained are too high for students that score perfectly in one of the tests but does mediocre on the others, compared to students that consistently score almost perfectly).

Since both the skill and difficulty parameters have to be estimated, it seems odd to me that one likelihood function works better for some parameters but not the others. Also there is the question of which function is better justified mathematically (maybe I'm doing something wrong, and the proper likelihood function is something else entirely).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.