9
$\begingroup$

all -

I'm trying to figure out if this can be done at all, and if so, how; unfortunately, my skill set isn't up to the task, and I'm hoping to find an answer here - or at least guidance on how to approach it.

I need to score a 10-question test that has weighted questions on a pass/fail basis, with 70% as the cut score. So far, so good - sum the total difficulties, multiply by 0.7, then check to see if the candidate's score is above or below (the framework we are using does all this automatically.) However, I now want to add another condition: if the candidate does a certain thing (something I can test for), I need the score to always be a fail (below 70%.)

Changing the scoring framework to do this would be problematic in lots of ways - but treating the condition test as a hidden 11th question would be trivial. The challenge, then, is this:

Given 10 questions with associated difficulty scores (as below), is it possible to calculate a difficulty score for question #11 such that failing it will produce a score below 70% in all cases, but passing it will result in the same pass/fail rate as if q#11 wasn't present? The weight can't be negative.

Hopefully, I've managed to phrase this in a coherent manner; please feel free to ask any questions you need to clarify.

Example of question weights:

  1. 2
  2. 3
  3. 2
  4. 2
  5. 3
  6. 2
  7. 1
  8. 3
  9. 2
  10. 3
  11. ???
$\endgroup$
  • 2
    $\begingroup$ Are there partial marks for a question or do you always get either 0 or the full mark. $\endgroup$ – FullofDill Sep 14 '17 at 20:10
  • $\begingroup$ @FullofDill It's pass/fail, no partial scores - so 69.999% is a fail, while 70% is a pass. $\endgroup$ – bluewater_sailor Sep 14 '17 at 20:13
  • $\begingroup$ The whole test is pass/fail, but what about individual questions? For instance, can I score a '1' on question 1 or only 0/2? $\endgroup$ – FullofDill Sep 14 '17 at 20:15
  • $\begingroup$ Only 0 or 2 - no partials for individual questions either. $\endgroup$ – bluewater_sailor Sep 14 '17 at 20:18
  • $\begingroup$ @bluewater_sailor Can you change the threshold? Could you make it 80% instead of 70%? $\endgroup$ – Théophile Sep 14 '17 at 20:22
4
$\begingroup$

No it can't be done. For simplicity we scale all the scores such that the original ones sum to 1. Now we define x to be the weight of question 11 so the new total score is $1 + x$. Then for a student who gets all questions but 11 right to still fail we need $$1 < 0.7(1+x) \iff x > 3/7 \approx 0.43$$ but for a student with 60% on the first 10 questions and question 11 right to also fail we need $$\frac{0.6+x}{1+x} < 0.7 \iff x < 1/3 \approx 0.33$$ a contradiction.

Remark: This assumes it to be possible to score somewhere between 58% and 70% on the original test, which is certainly true for the scores you posted: Answering only the first 6 questions correctly will get you 60%

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ THAT is just perfect. I couldn't even figure out how to reason about the numbers here, and you've just demonstrated it. Awesome! $\endgroup$ – bluewater_sailor Sep 14 '17 at 20:46
  • $\begingroup$ \iff $\iff$ and \implies $\implies$ are usually preferred to those little stub arrows ;) $\endgroup$ – gen-z ready to perish Sep 14 '17 at 22:04
1
$\begingroup$

Note: in what follows I am treating $70$ as an absolute score. I believe the conditions of the post can not be satisfied as written, but if you let $70$ be an absolute score instead of a proportion of the total you can do it.

Let question $11$ be worth $31$ and let the other ten questions sum to $55.7$ (so $5.57$ each).

Then if you miss $11$ your maximum score is $69$ so you are sure to fail.

In order to pass you need question $11$ plus $70\%$ of the rest. Thus the rest has to have value $$100\times \left(1-\frac {.31}{.7}\right)\approx 55.714$$ It follows that, having gotten $11$ right you need to pass the test in the ordinary way.

Note: with these weights, your maximum score is $86.7$. Not sure if that's a problem.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Is there a way to turn this around? In other words, given an arbitrary set of weights for the different questions, how would I calculate the weight for #11? $\endgroup$ – bluewater_sailor Sep 14 '17 at 20:29
  • $\begingroup$ Sure. If the total is of the rest is $X$, so $X=55.7$ in my scheme, then the score for $\#11$ is $70\times \left(1-\frac X{100}\right)$ $\endgroup$ – lulu Sep 14 '17 at 20:32
  • $\begingroup$ Is this the sort of thing you were looking for? $\endgroup$ – lulu Sep 14 '17 at 20:32
  • $\begingroup$ I'm going to try it out with some score scenarios, but - yes, thank you! That is exactly what I was looking for! $\endgroup$ – bluewater_sailor Sep 14 '17 at 20:33
  • $\begingroup$ Well, I'm not sure it's what you want. I was working with $70$ as an absolute score, not a ratio. I don't think my approach works with the ratio. $\endgroup$ – lulu Sep 14 '17 at 20:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.