So after all, we put together a little sample script for simulation:
We implemented the naive linear approach and two modified version of binary search:
- synalgo performs simple binary search and tries to estimate the expected remaining cost. If the expected cost of moving on with binary search is higher than the expected cost of the linear search it falls back to linear
- cj performs a modified binary search where the search space is split in the ration of the good answer/bad answer costs - in the case described in the questions the algorithm always asks if the number is in the lower 1/10 of the current interval. This algorithm also falls back to linear search when the 1/10th of the interval becomes less than 1
After 10000 runs with an upper bound of 100 the results look like this:
naive average of 10000 runs: 59.8012
synalgo average of 10000 runs: 32.9129
cj average of 10000 runs: 30.9006
CJ is usually ~10% better than synalgo and they both present good results compared to the naive approach.
These algorithms are of course not proven (close-to) optimal, but provide acceptable efficiency for my actual problem. Any further optimization proposals are welcome though!