Suppose $\Theta \subset \mathbb{R}^d$ is a convex set, and $f:\Theta \rightarrow \mathbb{R}$ is a strictly convex function that has a minimum at $\theta_0\in\Theta$. Is it true then that $\forall \varepsilon>0,$ $$\, \inf_{\theta\notin B(\theta_0,\,\varepsilon)}f(\theta)> f(\theta_0),$$ where $B(\theta_0,\,\varepsilon)$ denotes the open ball of radius $\varepsilon$ with center $\theta_0$? If no, is there a counterexample? If yes, how would one prove that?
Many thanks for any help!
Yes. By strict convexity $\theta_0$ is the unique global minimum, as if $f(\theta_1)\leq f(\theta_0)$ then for any $0<t<1$ we have $$f(t\theta_0+(1-t)\theta_1)<tf(\theta_0)+(1-t)f(\theta_1)\leq f(\theta_0)$$ contradicting local minimality. For any point $\theta\notin B(\theta_0,\epsilon)$, we have $f(\theta)>f(\theta_0)$. Let $$\theta'=\theta_0+\frac{\epsilon}{\|\theta-\theta_0\|}(\theta-\theta_0)=\frac{\epsilon}{\|\theta-\theta_0\|}\theta+\left(1-\frac{\epsilon}{\|\theta-\theta_0\|}\right)\theta_0$$ and note that $0<\epsilon/\|\theta-\theta_0\|\leq 1$ so by convexity we get that $$f(\theta')\leq\frac{\epsilon}{\|\theta-\theta_0\|}f(\theta)+\left(1-\frac{\epsilon}{\|\theta-\theta_0\|}\right)f(\theta_0)\leq f(\theta).$$ Since $\theta'$ lies in the set $S=\{\theta:\|\theta-\theta_0\|=\epsilon\}\subseteq \Theta\setminus B(\theta_0,\epsilon)$, this gives us $\inf\limits_{\theta\notin B(\theta_0,\epsilon)}f(\theta)=\inf\limits_{\theta\in S}f(\theta)$. Since $S$ is compact it achieves its infimum, thus $\inf\limits_{\theta\in S}f(\theta)>f(\theta_0)$.
Edit: s_2 points out that this argument only works if $S\subseteq \Theta$, which may not be the case. To remedy this, note that it suffices to have $S\subseteq \Theta$ for sufficiently small $\epsilon$. If $S\nsubseteq \Theta$ for all $\epsilon$, then by convexity $\Theta$ is contained in an affine subspace of dimension less than $n$. Changing coordinates allows us to reduce the dimension, and we repeat this until $\Theta$ contains $S$ for some $\epsilon>0$, or we reach $n=0$, in which case $\Theta$ must be a single point and the result is trivial.
