# Is this linear optimal control problem uncontrollable?

If someone could help me with this problem I would be greatly appreciative.

Control the system $$\dot{x}=x+u$$ From $$x(0)=0 \space to\space x(T)=2$$ Where $T\in\mathbb{R}_+$ is free

s.t. $$J=\int_{0}^{T}\frac{1}{2}u^2dt$$ is minimised.

### My Approach

Let $$\\f_0(t,x,u)=\frac{1}{2}u^2 \\f_1(t,x,u)=x+u$$

The Hamiltonian of this problem is given by: $$\\H=-f_0+\psi \times f_1 \\=-\frac{1}{2}u^2+\psi(x+u)$$

By the PMP we wish to choose $u$ s.t. it maximises $H$, $$\\\frac{\partial H}{\partial u}=0 \\\Rightarrow\psi=u$$

The costate equation gives us $$\\\dot\psi= -\frac{\partial H}{\partial x} \\\Rightarrow\dot\psi=-\psi \\\Rightarrow\psi=Ae^{-t}$$

Subbing this back into the system gives $$\\x(t)=Be^{t}-\frac{A}{2}e^{-t} \\u(t)=Ae^{-t}$$

Now along the optimal trajectory, again by the PMP, $H$ must be $0$. As this applies along any point of the trajectory, we have (after a bit of algebra) $$\\H(t=0)=0 \\\Rightarrow A = 0 \space or \space B = 0$$

Now this is where I am confused, if $A=0$ or $B=0$ we have

$$x(t) = -\frac{A}{2}e^{-t} \space or \space x(t) = Be^{t}$$

But given $x(0)=0$ that would imply that in both cases $x(t)=0$. Which clearly does not give the optimal solution as it will never reach $x(T)=2$.

I'm not sure if I have made some fundamental error along the way, or if the system is just not controllable, but would appreciate some guidance either way.

• You made a mistake with $H = -f_0 + \psi \cdot f_1$, which should have been $H = f_0 + \psi \cdot f_1$. – Kwin van der Veen Nov 2 '17 at 23:52
• Could you elaborate please @KwinvanderVeen? As I understand it, the PMP states that $\psi_0 < 0$ and constant. In my course we have always just set it to -1. – CptB3RRY Nov 3 '17 at 0:13
• $\psi$ is not constant and you stated that $\psi(t)=A\,e^{-t}$. But you are focussing of $\psi$, but not on the minus sign in front of $f_0$. – Kwin van der Veen Nov 3 '17 at 0:55
• Sorry, miscommunication on my part there @KwinvanderVeen. If we write $H=\psi_0 f_0 + \psi_1 f_1$ then as per my approach $\psi_0 = -1$, which as I understand it is what it is normally set to. – CptB3RRY Nov 3 '17 at 1:08
• I have never seen such $\psi_0$. I only know it as the same notation used on Wikipedia, so with $\lambda$, the costate, instead of $\psi$ (or $\psi_1$). Maybe that negative number has be applied if you want to maximize a profit function, instead of minimizing a cost function. – Kwin van der Veen Nov 3 '17 at 2:00

OK, you started well, but then something went wrong. You've found $u(t)=\psi(t)$. That's right. The equation for $\psi(t)$ is correct either. I'll just write it as $\psi(t)=\psi_0 e^{-t}$, where $\psi_0=\psi(0)$.
Then you substitute your $u(t)=\psi(t)$ into the DE for $x$, i.e., $\dot{x}=x+\psi_0e^{-t}$, and solve it to get $$x(t)=x_0e^t + \int_0^t e^{t-\tau}\psi_0 e^{-\tau}d\tau=x_0 e^t + \psi_0 \frac{e^t-e^{-t}}{2}.$$ Taking into account that $x(0)=0$ and using some hyperbolic trigonometry notation we obtain $$x(t)=\psi_0 \sinh(t).$$ It remains to sustitute the final time $T$ and solve the preceding equation to get $\psi_0=\frac{2}{\sinh(T)}$ whence you can compute the optimal control etc.
ADDED: Assume now that $T$ is free. We can compute the cost function $J=\frac{4}{e^{2T}-1}$, which attains minimum at $T\to \infty$ which is equivalent to $\psi_0=0$ and hence, $u(t)=0$. This implies that the problem does not have a solution.
It is interesting that for $T>5$ the cost $J$ becomes infinitesimally small, so one can get a practically optimal solution by setting $T$ to an arbitrary constant larger than 5. But this is a different story.
• Your analysis seems correct but there is one point that appears inconsistent. If we replace $u$ and $\psi$ in the Hamiltonian we get $H(t)=\frac{\psi_0^2}{2}$ for all $t$. Shouldn't the Hamiltonian be equal to zero as this is a free-final time problem? But if $\psi_0=0$ then the state cannot reach $x(T)=2$. In my opinion the problem is ill-posed and there does not exist an optimal control law. – RTJ Nov 3 '17 at 17:18
• It wasn't said explicitly that $T$ is free. But you are right, of course. I updated the answer accordingly. – Dmitry Nov 3 '17 at 18:03
• You are right in that he did not stated explicitly that $T$ is free. But this was stated implicitly in his attempt to find the optimal control where he used $H=0$. Anyway, you answer seems complete now so (+1). – RTJ Nov 3 '17 at 18:15
• Sorry, I should indeed have stated that $T$ is free and have amended the question accordingly. Your solution for $x(t)$ is actually what I got the first time around when I considered $x(0) =0$ prior to considering $H=0$. But regardless, your explanation makes sense and is very helpful, thank you. – CptB3RRY Nov 4 '17 at 0:14