# Understanding a “Trivial” Result In Optimal Control Theory

In Lawrence C. Evans' online notes : Optimal control theory, page 33, Evans makes a very trivial looking statement,which doesn't seem trivial to me. I shall elaborate, giving necessary details, so that this reference is only for further reading.

Let $\alpha : [0, \infty) \to [-1,1]^m$ be a measurable function. Consider the differential equation: $$\dot x(t) = M x(t) + N\alpha(t) \\ x(0) = x_0$$

Where $M,N$ are constant real matrices of appropriate dimension, so that $x(t)$ has codomain $\mathbb{R}^n$. Now, we can solve this equation, with the solution: $$x(t) = X(t)x_0 + X(t) \displaystyle\int_0^t X^{-1}(s) N\alpha(s) \operatorname{ds}$$

Now, fix a time $T$, and define the following set: $$K(T,x_0) = \left\{ X(T)x_0 + X(T) \displaystyle\int_0^T X^{-1}(s) N\alpha(s) \operatorname{ds}\right\}$$ where the $\alpha$ can vary over all measurable functions possible from $[0,\infty) \to [-1,1]^m$.

In words, we are trying to find all reachable points at time $T$. If $x_1 \in K(T,x_0)$, there exists $\alpha_{x_1}$ such that $x_1 = X(T)x_0 + X(T) \displaystyle\int_0^T X^{-1}(s) N\alpha_{x_1}(s) \operatorname{ds}$.

One can prove that $K(T,x_0)$ is convex and closed, the former using an obvious candidate, and the latter via an application of the Banach-Alaoglu theorem.

Now, suppose that $0 \in K(\tau, x_0)$ for some $\tau$, but it is not true that $0 \in K(t, x_0)$ for all $t < \tau$. I want to prove that $0$ is a boundary point of $K(\tau,x_0)$.

In words: "If $0$ is reachable in time $\tau$, but is not reachable in any time smaller than $\tau$, then $0$ is a boundary point of the set of all points reachable in time $\tau$".

It did not seem trivial, so I tried letting $0$ be an interior point. It is clear that in a neighbourhood of $0$, there are points which can be reached in time $< \tau$ (when we take the trajectory from $x_0$ to $0$, it enters the neighbourhood before time $\tau$, so all points falling on that trajectory are reachable in time $< \tau$, naturally).However, I am unable to show why there must be points which are not reachable in time $\tau$ in a neighbourhood of $0$. This is because I am unable to use the nature of the given set to my benefit.

Surely if the question is trivial, then I am missing something. Do guide me across this one.

• Introduce an additional state variable, $y$, which behaves like time: $$\dot{y} = 1.$$ Now your set of all states reachable in time $T$ is simply the subset $y=\tau$ of the new state space. Now show that $y=\tau$ is a sufficiently smooth hypersurface in your (new) state space (hint: it is a hyperplane:), which "cuts" the neighborhood of $0$ you mentioned into 2 parts. – avs Feb 7 '17 at 0:32
• @avs Yes, I agree with you. However, I am unable to see why it should be a hyperplane. $y = \tau$ is cleary a hyperplane in the state space, however the transformation back to the original space may not be linear, right?(If what you are saying is right,which it should be, then we are done). Please excoriate me if I am making any comical/obvious statements in this comment. – астон вілла олоф мэллбэрг Feb 7 '17 at 0:38
• I may be mistaken (sorry, don't have time to get into it), but I was thinking, the transformation back to the original space is the orthogonal projection onto the $x$-coordinates. If so, this projection is a linear transformation. – avs Feb 7 '17 at 0:53
• @avs I'm unable to prove that's true, because the nature of "reachability" doesn't give us the flexibility to say the transformation is linear, because $\alpha$ is only given to be measurable. If you can come up with a more definitive explanation (that is, whenever you have the time to get to it) then I would be happy to hear. I do not wish to distract you, and sometimes a discussion is more enlightening than an answer. Thank you. – астон вілла олоф мэллбэрг Feb 7 '17 at 0:57
• I would experiment with some examples with $n \leq 2$ and $m \leq 2$, just to get insight. $\alpha$ parameterizes the vector fields on the state space, not the state space itself (sorry if I am saying what you already knew). Good luck. – avs Feb 7 '17 at 1:15

It is easier to argue if we introduce the following set $$R_T=\left\{\int_0^T X^{-1}(s) N\alpha(s)\,ds\right\}.$$ Then $$K(T,x_0)=X(T)(x_0+R_T)$$ and it is quite obvious that \begin{align} 0\not\in K(t,x_0)\quad&\Leftrightarrow\quad -x_0\not\in R_t,\\ 0\in \partial K(\tau,x_0)\quad&\Leftrightarrow\quad -x_0\in \partial R_{\tau}. \end{align} The sets $R_t$ are convex and closed as well, but they are ordered by inclusion as $$t_1\le t_2\quad\Rightarrow\quad R_{t_1}\subset R_{t_2}$$ since $\alpha=0$ is allowed. The sets are also continuous: for any $\epsilon>0$ there exists $t<\tau$ such that $$R_t\subset R_{\tau}\subset R_t+\epsilon B_1$$ where $B_1$ is the unit ball. It is because the integral $$\int_0^t X(s)^{-1}N\alpha(s)\,ds$$ is continuous in $t$ (actually equicontinuous for all $\alpha$ in our class).

To prove that $-x_0$ is the boundary point we can take a ball $B$ around it of an arbitrary radius $\mu>0$ and show that there are points in it that are outside of $R_{\tau}$.

The sets $R_t$ are convex and closed. Moreover, the center point $-x_0\not\in R_t$ for $t<\tau$, so one can separate the center of the ball from $R_t$ by a hyperplane. Thus the volume of $R_t$ inside the ball is less than half of the ball volume. By continuity we can find $t<\tau$ such that $$R_{\tau}\subset R_t+\frac{\mu}{4}B_1$$ and $R_t+\frac{\mu}{4}B_1$ not filling up the whole ball $B$, leaving the top quarter for points outside $R_{\tau}$.

• Thank you very much for your answer! I shall get back to you if I have any doubts. – астон вілла олоф мэллбэрг Mar 9 '17 at 23:26
• Here is my first doubt: Why is it obvious that $0 \in \partial K(\tau, x_0) \iff -x_0 \in \partial R_\tau$? – астон вілла олоф мэллбэрг Mar 10 '17 at 3:07
• The mapping $x\mapsto X(x_0+x)$ with an invertible $X$ is a homeomorphism, so a neighbourhood of $-x_0$ maps to a neighbourhood of $0$ and vice versa. It gives that if $-x_0$ is an interior point of $R$ then $0$ is an interior point of $K$, if $-x_0$ is a interior point of ${\Bbb R}^n\setminus R$ then $0$ is an interior point of ${\Bbb R}^n\setminus K$ and so on. – A.Γ. Mar 10 '17 at 5:09
• Yes, that's right. Thank you for your excellent response to this question, in honesty I have been struggling with it for some time. I shall get back to you in some time if I have further questions. – астон вілла олоф мэллбэрг Mar 11 '17 at 3:10
• Your questions are welcome. I'm used to work with control, but it has passed some time. – A.Γ. Mar 11 '17 at 7:37

Sorry, I'm not an expert. However, I try to give you my insight with the hope that it helps.
In order to check whether $0$ is a boundary point of $K(\tau,x_0)$, it suffices to show that any neighbourhood of $0$ contains a point in $K(\tau,x_0)$ and a point outside. Let's fix $\epsilon>0$ and take the open ball $B:=\{x\in\mathbb R^n \,:\, \|x\|<\epsilon\}$. Since $0\in K(\tau,x_0)$, there exists $\alpha:[0,\tau]\to[-1,1]^m$ so that $$0 = X(\tau)x_0 + X(\tau)\int_0^\tau X^{-1}(s)N\alpha(s)ds$$ For any $T<\tau$ take the state $$x(T) = X(T)x_0 + X(T)\int_0^T X^{-1}(s)N\alpha(s)ds$$ Then $x(T)$ is in $K(\tau,x_0)$, since it is in $K(T,x_0)$.
Let's choose $T$ so that $x(T)\in B$. That is possible, since, once fixed $x_0$, $t\mapsto x(t)$ is differentiable and thus locally Lipschitz. Hence we can bound $\|x(\tau)-x(T)\|$ as $$\|x(\tau)-x(T)\|\le L\|\tau-T\|$$ with $L$ that depends on $\tau$. We can thus chose $T$ such that $\|x(\tau)-x(T)\|<\epsilon$. This proves that in the $\epsilon$-neighbourhood of $x(\tau)$ there exists a point in $K(\tau,x_0)$. Take now $T>\tau$ and $\alpha'(s)$ that is any properly defined non-zero function that agrees with $\alpha$ on $[0,\tau]$. then $x(T)\notin K(t,x_0)$, for all $t\le \tau$, otherwise $x(\tau)$ would be reachable in less that $\tau$ seconds, that violates the hypothesis. Using the same argument as before you can chose $T>\tau$ so that $x(T)$ is in the $\epsilon$-neighbourhood of $x(\tau)$, thus proving that in such neighbourhood there exists a point that does not belong to $K(\tau,x_0)$. At this point, the claim should follow from the arbitrariness of $\epsilon$.
Maybe this is not a rigorous proof, but I hope that it can be an idea.

• It may not be rigorous, but I think the answer qualifies as an excellent one nevertheless. Thank you very much for your input. – астон вілла олоф мэллбэрг Feb 8 '17 at 23:16
• Cool. Glad it helped – LJSilver Feb 8 '17 at 23:17
• You are welcome! – астон вілла олоф мэллбэрг Feb 8 '17 at 23:18
• It is not necessarily true that $K(T,x_0)$ is a subset of $K(τ,x_0)$ for $T<τ$, However $X(T)^{-1}K(T,x_0)\subset X(τ)^{-1}K(τ,x_0)$. – LutzL Feb 9 '17 at 16:16
• @LutzL I'm responding late, my apologies. Why is it not true that $K(T,x_0) \subset K(\tau,x_0)$? We can just extend the control which steers to $0$ at time $T$, by a constant function zero. From the equation described, you can see that this doesn't affect the final constant. – астон вілла олоф мэллбэрг Feb 16 '17 at 6:47