If someone could help me with this problem I would be greatly appreciative.

Control the system $$\dot{x}=x+u$$ From $$x(0)=0 \space to\space x(T)=2$$ Where $T\in\mathbb{R}_+$ is free

s.t. $$J=\int_{0}^{T}\frac{1}{2}u^2dt$$ is minimised.

My Approach

Let $$ \\f_0(t,x,u)=\frac{1}{2}u^2 \\f_1(t,x,u)=x+u $$

The Hamiltonian of this problem is given by: $$ \\H=-f_0+\psi \times f_1 \\=-\frac{1}{2}u^2+\psi(x+u) $$

By the PMP we wish to choose $u$ s.t. it maximises $H$, $$ \\\frac{\partial H}{\partial u}=0 \\\Rightarrow\psi=u $$

The costate equation gives us $$ \\\dot\psi= -\frac{\partial H}{\partial x} \\\Rightarrow\dot\psi=-\psi \\\Rightarrow\psi=Ae^{-t} $$

Subbing this back into the system gives $$ \\x(t)=Be^{t}-\frac{A}{2}e^{-t} \\u(t)=Ae^{-t} $$

Now along the optimal trajectory, again by the PMP, $H$ must be $0$. As this applies along any point of the trajectory, we have (after a bit of algebra) $$ \\H(t=0)=0 \\\Rightarrow A = 0 \space or \space B = 0 $$

Now this is where I am confused, if $A=0$ or $B=0$ we have

$$x(t) = -\frac{A}{2}e^{-t} \space or \space x(t) = Be^{t} $$

But given $x(0)=0$ that would imply that in both cases $x(t)=0$. Which clearly does not give the optimal solution as it will never reach $x(T)=2$.

I'm not sure if I have made some fundamental error along the way, or if the system is just not controllable, but would appreciate some guidance either way.

  • $\begingroup$ You made a mistake with $H = -f_0 + \psi \cdot f_1$, which should have been $H = f_0 + \psi \cdot f_1$. $\endgroup$ – Kwin van der Veen Nov 2 '17 at 23:52
  • $\begingroup$ Could you elaborate please @KwinvanderVeen? As I understand it, the PMP states that $\psi_0 < 0$ and constant. In my course we have always just set it to -1. $\endgroup$ – CptB3RRY Nov 3 '17 at 0:13
  • $\begingroup$ $\psi$ is not constant and you stated that $\psi(t)=A\,e^{-t}$. But you are focussing of $\psi$, but not on the minus sign in front of $f_0$. $\endgroup$ – Kwin van der Veen Nov 3 '17 at 0:55
  • $\begingroup$ Sorry, miscommunication on my part there @KwinvanderVeen. If we write $H=\psi_0 f_0 + \psi_1 f_1$ then as per my approach $\psi_0 = -1$, which as I understand it is what it is normally set to. $\endgroup$ – CptB3RRY Nov 3 '17 at 1:08
  • $\begingroup$ I have never seen such $\psi_0$. I only know it as the same notation used on Wikipedia, so with $\lambda$, the costate, instead of $\psi$ (or $\psi_1$). Maybe that negative number has be applied if you want to maximize a profit function, instead of minimizing a cost function. $\endgroup$ – Kwin van der Veen Nov 3 '17 at 2:00

OK, you started well, but then something went wrong. You've found $u(t)=\psi(t)$. That's right. The equation for $\psi(t)$ is correct either. I'll just write it as $\psi(t)=\psi_0 e^{-t}$, where $\psi_0=\psi(0)$.

Then you substitute your $u(t)=\psi(t)$ into the DE for $x$, i.e., $\dot{x}=x+\psi_0e^{-t}$, and solve it to get $$x(t)=x_0e^t + \int_0^t e^{t-\tau}\psi_0 e^{-\tau}d\tau=x_0 e^t + \psi_0 \frac{e^t-e^{-t}}{2}.$$ Taking into account that $x(0)=0$ and using some hyperbolic trigonometry notation we obtain $$x(t)=\psi_0 \sinh(t).$$ It remains to sustitute the final time $T$ and solve the preceding equation to get $\psi_0=\frac{2}{\sinh(T)}$ whence you can compute the optimal control etc.

ADDED: Assume now that $T$ is free. We can compute the cost function $J=\frac{4}{e^{2T}-1}$, which attains minimum at $T\to \infty$ which is equivalent to $\psi_0=0$ and hence, $u(t)=0$. This implies that the problem does not have a solution.

It is interesting that for $T>5$ the cost $J$ becomes infinitesimally small, so one can get a practically optimal solution by setting $T$ to an arbitrary constant larger than 5. But this is a different story.

  • $\begingroup$ Your analysis seems correct but there is one point that appears inconsistent. If we replace $u$ and $\psi$ in the Hamiltonian we get $H(t)=\frac{\psi_0^2}{2}$ for all $t$. Shouldn't the Hamiltonian be equal to zero as this is a free-final time problem? But if $\psi_0=0$ then the state cannot reach $x(T)=2$. In my opinion the problem is ill-posed and there does not exist an optimal control law. $\endgroup$ – RTJ Nov 3 '17 at 17:18
  • $\begingroup$ It wasn't said explicitly that $T$ is free. But you are right, of course. I updated the answer accordingly. $\endgroup$ – Dmitry Nov 3 '17 at 18:03
  • 1
    $\begingroup$ You are right in that he did not stated explicitly that $T$ is free. But this was stated implicitly in his attempt to find the optimal control where he used $H=0$. Anyway, you answer seems complete now so (+1). $\endgroup$ – RTJ Nov 3 '17 at 18:15
  • 1
    $\begingroup$ Sorry, I should indeed have stated that $T$ is free and have amended the question accordingly. Your solution for $x(t)$ is actually what I got the first time around when I considered $x(0) =0$ prior to considering $H=0$. But regardless, your explanation makes sense and is very helpful, thank you. $\endgroup$ – CptB3RRY Nov 4 '17 at 0:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.