Student has ${1\over{h+1}}$ chance of being wrong and teacher has ${1\over{k+1}}$ chance of grading wrong sum as actually wrong Here's a question from my probability textbook:

A boy has done $n$ sums; the odds are $h : 1$ against any given sum being wrong; and the odds are $k: 1$ against the examiner seeing that any given wrong sum is wrong. If the examiner discovers that $r$ sums are wrong, show that the most likely number to be wrong is the greatest integer in $(rh + nk + rhk) \div (h + k + hk)$.

Here's what I did. Starting from $n$ sums, the probability the boy gets $x$ wrong is $\binom{n}{x} {{h^{n-x}}\over{(h + 1)^n}}$. And assuming the boy gets $x$ wrong, the probability of the examiner discovering $r$ wrong is $\binom{x}{r} {{k^{x - r}}\over{(k+1)^x}}$. So we want to find $x$ such that$$\binom{n}{x} {{h^{n-x}}\over{(h + 1)^n}} \times \binom{x}{r} {{k^{x - r}}\over{(k+1)^x}} = {{h!h^{r - x}k^{x - r}}\over{(n - x)!(x - r)! r! (h + 1)^n (k + 1)^x}}$$is maximized. But I'm not sure on how to go about doing that. Any help would be appreciated.
 A: Hint
Call that last expression you wrote $f(x)$. You want to find the $x$ maximizing $f(x)$.
To do this, consider this ratio $f(x)/f(x-1)$. The optimal  $x$ is the largest value of $x$ for which $f(x)/f(x-1)>1$. Indeed, if $x^*$ is that largest value, then you have $f(x^*)/f(x^*-1)>1$, while $f(x^*+1)/f(x^*)<1$. This implies $$f(x^*-1)<f(x^*)>f(x^*+1),$$ meaning $x^*$ is a local maximum. It turns out there are no other local maxima, so $x^*$ is the global maximum.
Fortunately, the ratio $f(x)/f(x-1)$ greatly simplifies, so it is easy to determine $x^*$. Try it out!
This same technique tends to be very useful whenever you are optimizing functions $f:\mathbb N\to \mathbb R$ that are products of exponentials and factorials.

One minor nitpick; the expression you computed is the unconditional probability that the student makes $x$ mistakes and that the examiner notices $r$ of them. The problem is implicitly asking you maximize the conditional probability of $x$ mistakes given that the teacher notices $r$ of them. To correct your expression, you would have to divide by the (unconditional) probability that the teacher notices $r$ mistakes. However, this does not affect the computation above, since you would by dividing the expression you had by a constant independent of $x$.
To find the unconditional probability of $r$ noticed mistakes, you choose the $r$ problems that will be noticed mistakes in $\binom{n}r$ ways, and then you multiply by $p^r(1-p)^{n-r}$, where $p$ is the probability that a particular problem is a noticed mistake. You can express $p$ in terms of $h$ and $k$.
A: I am going to use the fact that if $X \sim \mathcal B(N, p)$ then the most likely number taken by $X$ is $\lceil Np\rceil$. Let start by doing some notations : $U_i$ is the binary random variable that the student gets the sum $i$ wrong and $V_i$ is the binary random variable that the teacher sees that the student gets the sum $i$ wrong. You can see that :

*

*The probability that the student gets a sum wrong is $\frac1{1+h}$ :
$$P\left[U_i = 1\right]=\frac{1}{1+h}$$


*When the student gets the sum right the teacher can not see that he gets it wrong : $$P\left[V_i = 1\Big| U_i = 0\right] = 0$$


*When the student gets the sum wrong the probability that the teacher sees that he gets wrong : $$P\left[V_i = 1\Big | U_i = 1\right] = \frac{1}{k+1}.$$
So if we note $X = \sum_{i=1}^n U_i$ and $Y = \sum_{i=1}^n V_i$, you are looking for $x$ such that $$P\left[X = x \left|Y = r\right.\right]$$ is maximum. Since computations of the sums are independants and using then $$P\left[\left.\sum_{i=1}^{n-r} U_i = x-r\right|V_k = 0; k=1,\ldots,n-r\right] = \binom{n-r}{x-r}p^{x-r}(1-p)^{n-x}.$$
Where \begin{align}
p &= P\left[U = 1\Big | V=0\right]\\
&=\frac{P\left[V=0\Big | U=1\right]P\left[U=1\right]}{P\left[V=0\right]}\\
&= \frac{\frac{k}{k+1}\cdot\frac{1}{h+1}}{\frac{k}{k+1}\cdot\frac{1}{h+1} + 1\cdot\frac{h}{h+1}}\\
&= \frac{k}{k+h+hk}.
\end{align}
So $$X_{\Big | Y =r} = r + Z$$ with $Z \sim\mathcal B\left(n-r, \frac{k}{k+h+hk}\right).$ Using the fact that I state first you are looking for $$x = r + \left\lceil (n-r)\frac{k}{k+h+hk}\right\rceil.$$ Which is exactly what you are looking for.
