An object is moving with accordance to Newton's laws: $\begin{pmatrix} \dot{y} \\ \dot {v} \end{pmatrix} = \begin{pmatrix} v \\ u \end{pmatrix}$ where $y$ is the objects location and $v$ is its speed.

Let $x=\begin{pmatrix} y \\ v \end{pmatrix}$ be the state variable. Assume our input controller (the acceleration) $u$ is such that $|u| \leq 1$.

Using Jacobi-Hamilton (Hamiltonian) method, find an optimal controller that shifts $y(0),v(0)$ to the origin $(0,0)$ in minimal time $T$.

What I did:

Our cost function is $J(0)=\int_{0}^{T}1d\tau$ and our state equation is $\dot{x} = \begin{pmatrix} \dot{y} \\ \dot {v} \end{pmatrix} = \begin{pmatrix} v \\ u \end{pmatrix}$

And so our Hamiltonian function is $H(x,u,\lambda) = 1+\lambda^ T\dot{x}$

and if $\lambda = \begin{pmatrix} \lambda _1 \\ \lambda _2 \end{pmatrix}$ then $H=1+\lambda _1 v+\lambda _2 u$

Please note - This is also the result the teacher reached. So I doubt this is incorrect.

The method says that $\dot{x} = H_\lambda$, $\dot {\lambda} = -H_x$, and $H_u = 0$.

From the first equation we get $\dot{x} =\begin{pmatrix} \dot{y} \\ \dot {v} \end{pmatrix}=\begin{pmatrix} v \\ u \end{pmatrix}$ which we already know and is trivial.

From the second equation (I think) we get $\dot {\lambda} = \begin{pmatrix} \dot{\lambda _1 } \\ \dot {\lambda _2} \end{pmatrix}=-\begin{pmatrix} 0 \\ \lambda _1 \end{pmatrix}$ which is also problematic by itself since this is not solvable, but this isn't my main problem.

My main problem is with $H_u=0$. this means that $\lambda _2 =0$. So where does the $u$ part come in? I'm supposed to find an optimal controller $u$, but nothing is dependent on $u$. it does not appear anywhere.

Would appreciate any help, I can't understand the teachers solution either.


This is a standard example considered in advanced optimal control problems dealing with minimum time problems under input constraints. If you are a control engineer I suggest the book of Athans and Falb on optimal control that has several solved cases for this type of problems.

I will provide an outline of the solution. Since you have input constraints you cannot take $H_u=0$ to find the optimal controller. In fact the optimal controller evolves along the boundary of the set of admissible controls ($|u|\leq 1$). In this case Pontryagin's minimum (maximum) principle has to be used that states that the input should be selected from the set of admissible inputs and minimizes $H$. The optimal control law is therefore $$u=-sign(\lambda_2)$$ Since $\dot{\lambda}_1=0$, $\lambda_1$ is a constant, i.e. $\lambda_1(t)=\lambda_{10}$. Also, we have $\dot{\lambda}_2=-\lambda_1=-\lambda_{10}$ which yields $\lambda_2(t)=\lambda_{20}-\lambda_{10}t$. This means that $\lambda_2$ is a linear function of time $t$ and therefore can only change sign up to one time. As a result the control input is either +1 or -1 with at most one sign change.

In order to find the actual control input, one has to address the problem geometrically. You have to visualize the trajectories in the state space (for the two possible control inputs $\pm1$) for constant input. So assume $u=\delta=\pm 1$. Then, as long as the input does not change sign, $$v=v_0+\delta t$$ $$y=y_0+v_0t+\frac{\delta}{2}t^2$$ or parametrically in the state-space $$y=y_0+\delta v_0(v-v_0)+\frac{\delta}{2}(v-v_0)^2=y_0+\frac{\delta}{2}(v^2-v_0^2)$$ These points form parabolas in the state-space. From the desired set-point $(0,0)$ exactly 2 of these curves pass, namely $$y=\pm\frac{1}{2}v^2.$$ We are interested in the set $S=S_1\cup S_2$ with $S_1:=\{(y,v)| v\leq 0\: and \: y=v^2/2\}$ and $S_2:=\{(y,v)| v\geq 0\: and \: y=-v^2/2\}$. This is a curve that divides the space space in two disjoint regions $$R_1:=\{(y,v)|\big((y\geq 0)and(v>-\sqrt{2|y|})\big)or\big((y\leq 0)and(v>\sqrt{2|y|})\big)\}$$ $$R_2:=\{(y,v)|\big((y\geq 0)and(v<-\sqrt{2|y|})\big)or\big((y\leq 0)and(v<\sqrt{2|y|})\big)\}.$$

If $(y_0,v_0)\in S$ then no switching occurs and the optimal control law is $u(t) =-sign(v_0)$. If $(y_0,v_0)\notin S$ then either $(y_0,v_0)\in R_1$ or $(y_0,v_0)\in R_2$. In the first case we start with $u=-1$ until the state trajectory hits $S_1$ and then switch to $u=+1$ (to move along $S_1$). Alternatively, if $(y_0,v_0)\in R_2$ we start with $u=+1$ until the state trajectory hits $S_2$ and then switch to $u=-1$ (to move along $S_2$).

  • $\begingroup$ Thank you very much, good answer, very clear. $\endgroup$ – Oria Gruber Oct 5 '15 at 18:53
  • $\begingroup$ Nice outline, but I wonder how we know that this control is optimal when this is only a nessesary condtions? $\endgroup$ – user1 Feb 17 '18 at 13:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.