Optimizing a Chernoff bound with Bernstein's condition Let $X$ be a real valued, zero mean random variable satisfying berstein's condition with parameter $b$ so that the moment generating function satisfies:
$$\mathbb E\left [ \exp \left ( \lambda X\right )\right ]
\leq \exp \left ( \frac{\lambda^2\sigma^2/2}{1-\lambda b} \right )$$
for all $\lambda \in (0, 1/b)$ where $\sigma^2$ is the variance of $X$. Using this to evaluate a chernoff bound for $X$:
$$\mathbb P (X \geq t) \leq \inf_{\lambda \in (0, 1/b)} \exp \left ( \frac{\lambda^2\sigma^2/2}{1-\lambda b} -\lambda t\right )$$
All that remains is to choose an appropriate value of $\lambda$. The literature usually chooses $\lambda^* = t/(bt + \sigma^2)$ but it's not clear to me how this value is chosen or why it is a good choice.
Interestingly, the argument of the exponential is $0$ when $\lambda =  2t/(2bt + \sigma^2)$, which looks like the choice of $\lambda^*$ from the literature (but with a scaling on $t$). Is the process to identify $\lambda^*$ basically: 1. find a root, 2. choose a scaling of $t$ that minimizes the value of the RHS but still satisfies the constraint $\lambda \in (0, 1/b)$?
 A: I agree with what you say in your last paragraph. The formal approach is to note that since the upper bound holds for all $\lambda$ in the range $(0, 1/b)$, then we should choose the best possible $\lambda$ in this range, i.e. the one that minimizes the upper bound, $\exp(f(\lambda))$ where $f(\lambda) =  \frac{\lambda^2}{1-\lambda b} \frac{\sigma^2}{2} - \lambda t$, subject to the constraint.
Another, I guess 'less formal' approach, is to choose $\lambda$ so that the exponent is negative. Note that if $f(\lambda) \ge 0$, the bound is not very good since $\exp(f(\lambda)) \ge 1$, which is a trivial upper bound for any probability. If $f(\lambda)$ is negative, then as $t$ becomes larger the bound is exponentially small, which is desired. To this end, we need to choose $\lambda$ so that $f(\lambda) < 0$, and so
\begin{align*}
f(\lambda)<0 &\iff  \frac{\sigma^2}{2} \lambda^2 < \lambda t(1-\lambda b)\\
&\iff  \lambda ( 1+ \frac{2tb}{\sigma^2}) < \frac{2t}{\sigma^2} \\
&\iff  \lambda  < \frac{2t}{\sigma^2 + 2 tb} < \frac{t}{\sigma^2 + tb}.
\end{align*}
where the last inequality assumes $\sigma>0$. So we see that the argument is zero exactly at the worst case point $\frac{2t}{\sigma^2 + 2 tb}$ which we need to choose $\lambda$ to be smaller than. Note that for any $t$ and $\sigma>0$
$$
\frac{t}{\sigma^2 + tb} \le \frac{t}{tb} =\frac{1}{b}
$$
so this choice of $\lambda$ is valid.
