# Convexity and mean curvature

Let $N$ be a Riemannian 3-Manifold with and $M \subset N$ an embedded, oriented codimension $0$ submanifold-with-boundary, bounded by a non-empty smooth subsurface $S := \partial M$. Now, with $M$ and therefore $S$ being oriented, the latter has a tubular neighborhood $T \cong S\times(-1,1)$ via some diffeomorphism $\phi$, such that under the same $\phi$, $T \cap M \cong S\times(-1,0]$. We call this the inward normal section of $T$ and, for $s \in S$, its intersection with the fiber $F_s \subset T$ the inward normal at $s$.

Further, we say that $M$ has almost convex boundary if for any $s \in S$ there exists an $\epsilon > 0$ such that for all $0 < \delta < \epsilon$, the cut-off ball $B(\delta,s) \cap M$ is a convex subset of $N$ (any two points in this set can be connected by a minimal geodesic which lies completely inside the set). In contrast, $M$ is said to have sufficiently convex boundary if $S$ has nonnegative mean curvature with respect to the inward normal section. Now I want to prove that $$\mbox{M has almost convex boundary} \implies\mbox{M has sufficiently convex boundary}.$$ As it is natural (for me at least) with these kinds of problems, I first tried to solve this problem in eucledian space $\mathbb R^3$ and then look at what needs to be changed and generalized for arbitrary 3-manifolds. Here is what I came up with so far:

If $M \subset \mathbb R^3$ and $s \in S$, there is a small neighborhood $U \ni s$ and a smooth function $f: U \to \mathbb R$ that describes $U$ in terms of its level sets, i.e we have $U \cap M = f^{-1}[0,\infty)$ and $U \cap S = f^{-1}(0)$. This implies that the inward unit normal field of $S$ restricted to $U$ can be expressed by $$\frac{\nabla f}{|\nabla f|},$$ with the corresponding mean curvature just being the negative divergence of the above vectorfield.

Now assuming $M$ has almost convex boundary and $U$ has been chosen sufficiently small, the unique line segment $(1-t)x + ty, t \in [0,1]$ connecting any two given points $x,y \in S \cap U$ lies completley in $M \cap U$, which is equivalent to $g_y(t) := f((1-t)x + ty) \geq 0$. Since $g_y(0) = f(x) = 0$, we must have $0 \leq g'_y(0) = \nabla f|_x * (y-x)$ by the chain rule.

This nicely illustrates an intuitive property for manifolds with almost convex boundary: The angle between the inward normal at $x$ and any vector emanating from $x$, pointing in the direction of $y$, should be at most $90$ degrees for any $y$ on $S$ sufficiently close to $x$. Moreover, it seems to me that this "angle condition" should also suffice to prove the nonnegativity of mean curvature. Does anyone have an idea how to proceed from here?

I think I have found an answer, but I believe it needs some more improvement, as there are things I am not completely sure about. Any help would be appreciated.

So suppose that $M$ has almost convex boundary and there exists some $x \in S$ with the above mean curvature $H(x) < 0$. This implies that there exists a small neighborhood $U \ni x$ in $S$ with $H|_U < 0$. By the tubular neighborhood theorem, a small neighborhood $V \ni X$ in $N$ can be identified with $U \times (-\epsilon,\epsilon)$ under a diffeomorphism $\psi: U \times (-\epsilon,\epsilon) \to V$ with $V \cap M = \psi(U \times (-\epsilon,0))$ and $V \cap S = \psi(U \times \{0\})$. Moreover, the pullback-metric $\psi^*g$ takes the form $g_{y,t} \oplus dt^2$ for $y \in U$, $t \in (-\epsilon, \epsilon)$.

By choosing sufficiently small $\epsilon$, this means that the negativity of $H$ on $U \cong U \times \{0\}$ translates to the following: $$\forall y \in U, \forall t_0 \in (-\epsilon,\epsilon), \forall X \in T_yU, X \neq 0: \frac{d}{dt}g_{y,t}(X,X)|_{t=t_0} =:\frac{d}{dt}||X||^2_{g(y,t)}|_{t=t_0} < 0$$.

From this, we can firstly deduce the following intermediate result:

For any curve $c:[0,1] \to U \times (-\epsilon,\epsilon), c(t)=(c_1(t),c_2(t))$ with $c_2(t) \leq 0$, the 'flat' curve $\tilde{c}(t)=(c_1(t),0)$ satisfies $L(\tilde{c}) \leq L(c)$.

Proof: Since $c_2(t) \leq 0$ by assumption, we have $||\dot{c}_1(t)||_{g(c_1(t),c_2(t))} \geq ||\dot{c}_1(t)||_{g(c_1(t),0)}$ and hence $$L(c) = \int_0^1 \sqrt{||\dot{c}_1(t)||^2_{g(c_1(t),c_2(t))}+||\dot{c}_2(t)||^2}dt \geq \int_0^1 ||\dot{c}_1(t)||_{g(c_1(t),c_2(t))}dt \geq \int_0^1||\dot{c}_1(t)||_{g(c_1(t),0)}dt = L(\tilde{c})$$

Now recall that $U \times (-\epsilon,0]$ corrensponds to $V \cap U$, which is by our assumption a convex set (for small $\epsilon$). This means that, for two points $(x,0),(y,0) \in U \times \{0\}$, there exists a minimal geodesic segment $\gamma: [0,1] \to U \times (-\epsilon,0]$ between them. By the above observation, we can assume that $\Gamma := \gamma([0,1]) \subset U \times \{0\}$. We show that this cannot be true, by considering a one-parameter variation of $\gamma$ with fixed boundary and monotone decreasing energy. Such a variation is, for example, given by $C:[0,1] \times (-\epsilon,\epsilon) \to U \times (-\epsilon,\epsilon)$ with $C(s,t) := (\gamma_1(s),(s-s^2)t)$. Then $C(s,0) = \gamma(s)$, $C(0,t)=(x,0)$, $C(1,t)=(y,0)$, and we have $$\frac{d}{dt}|_{t=0} E(C(s,t))= \int_0^1 \frac{\partial}{\partial t}|_{t=0} [||\dot{\gamma}_1(s)||^2_{g(\gamma_1(s),(s-s^2)t)} + ((2s-1)t)^2]ds = \int_0^1 (s-s^2)\frac{\partial}{\partial t}||\dot{\gamma}(s)||^2_{g(\gamma(s),t)}|_{t=0} ds < 0.$$

However, I am not so sure about the last equality, I simply used the chain rule by intuition, which is a valid step whenever $g_{y,t}$ can be factorized in $f(t)g$. Is it also true for the general case ?