Relationship between optimal value and optimal point for strongly convex and smooth function For a $\mu$ strong convex and $L$ smooth function $f:\mathbf{R}^n \to \mathbf{R}$, assume its unique global optimal point is $x^*$.
Intuitively, if our current $x$ is very close to $x^*$, then the current function value should also be close to $f(x^*)$. And conversely, if our current function value is very close to $f(x^*)$, our current $x$ should also be close to $x^*$. I am wondering that is this intuition correct? If it is, can we prove it?
For a possible formal expression of this intuition, can we use $||x-x^*||_2$ to control $|f(x) - f(x^*)|$? And can we use $|f(x) - f(x^*)|$ to control $||x-x^*||_2$?
 A: This gives detail on my comments: We have by standard subgradient inequalities:
If $f:\mathbb{R}^n\rightarrow\mathbb{R}$ is $L$-smooth then:
$$ f(y)\leq f(x) + f'(x)^{\top}(y-x) + \frac{L}{2}||y-x||^2 \quad \forall x, y \in \mathbb{R}^n$$
If $f:\mathbb{R}^n\rightarrow\mathbb{R}$ is $\mu$-strongly convex then for any $x \in \mathbb{R}^n$ and any $f'(x)\in \partial f(x)$:
$$ f(y) \geq f(x) + f'(x)^{\top}(y-x) + \frac{\mu}{2}||y-x||^2 \quad \forall x, y \in \mathbb{R}^n$$

If $f:\mathbb{R}^n\rightarrow\mathbb{R}$ is both $L$-smooth and $\mu$-strongly convex, and if $x^*$ is a global min of $f$, then $f'(x^*)=0$ and so we obtain
\begin{align}
f(y)\leq f(x^*) + \frac{L}{2}||y-x^*||^2 \quad \forall y \in \mathbb{R}^n\\
f(y)\geq f(x^*) + \frac{\mu}{2}||y-x^*||^2 \quad \forall y \in \mathbb{R}^n
\end{align}
Since $f(y)\geq f(x^*)$ for all $y$, we have $f(y)-f(x^*)=|f(y)-f(x^*)|$ and so the above two inequalities can be written
$$ \frac{\mu}{2}||y-x^*||^2\leq |f(y)-f(x^*)|\leq \frac{L}{2}||y-x^*||^2 \quad \forall y \in \mathbb{R}^n$$
From this it also follows that $L\geq \mu$.
