I'm having trouble with the following problem:

Control the following 1st order system $$\dot x = -x + u$$ from $x(0)=2$ to $x(4)=5$ such that the following cost function is minimized $$J= x(4) + \int_{0}^{4} u^2 (t) \, \mathrm{d}t$$ Find the optimal control $u^*$ and the optimal trajectory $x^*$.

I used the Pontryagin's maximum principle by writing the Hamiltonian

$$H = u^2 + \lambda ( - x + u)$$

Unfortunately, I did not find a stable solution ($u$ does not converge)

$$\frac{\partial H }{\partial u} =0 $$

implies $u^* = -\lambda/2$ and $\lambda$ is given by solving

$$\dot \lambda = - \frac{\partial H }{\partial x}$$

I found $\dot \lambda = \lambda$ which is first-order. The solution is an increasing exponential and, therefore, the control $u^*$ is an increasing exponential as well. Any help would be appreciated. Thanks!

  • 1
    $\begingroup$ Why do you expect that the control $u$ should converge? You are dealing with finite time optimal control. So as long as the calculated optimal $u^*$ remains bounded during that interval it should still be a valid solution. $\endgroup$ – Kwin van der Veen Jan 4 '18 at 7:17
  • 2
    $\begingroup$ If $x(4)$ is fixed, why is it in the cost function? $\endgroup$ – Rodrigo de Azevedo Jan 4 '18 at 9:20

Since you minimize your cost functional, the Pontryagin maximum principle says that the Hamiltonian is to be chosen as $H(x,\lambda,u,t)=\lambda_0 u^2(t) + \lambda(-x+u)$, where $\lambda_0$ can be set to $-1$ (we assume that the optimal control problem is regular). So, we have
$$H(x,\lambda,u,t)= -u^2 + \lambda(-x+u).$$ As @Rodrigo-de-Azevedo rightfully noticed, $x(4)$ does not play any role in the cost functional as the right end-point is fixed.

Now we use the first order optimality condition to determine $u^*=\lambda/2$. One can check that $u^*$ does indeed maximizes $H(x,\lambda,u)$ as $\frac{\partial^2 H}{\partial^2 u}=-2<0$, which implies that $H$ is concave w.r.t. $u$. As for now, we assume that $u^*\in [u_{min}, m_{max}]$, i.e., $u^*$ is within the admissible control set.

Now we can write the DE for $\lambda$, $\dot{\lambda}=\lambda$, which is not very informative as we do not know both the initial and the final values of $\lambda(t)$. So, we choose $\lambda(0)=\Lambda$ and try to find $\Lambda$ s.t. the end-point conditions on $x(t)$ are fulfilled.

Solving $\dot{\lambda}=\lambda$ with $\lambda(0)=\Lambda$ we get $\lambda(t)=\Lambda e^{t}$. Using $u^*=\lambda/2$ and substituting it into the DE for $x$ we get $$\dot{x}=-x+\Lambda e^t/2,\quad x(0)=2$$ which solves to yield $x(t)=\frac{1}{4}\left[\Lambda e^t - e^{- t}(\Lambda - 8)\right]$. Finally, from $x(4)=5$ we get $\Lambda=\dfrac{20 e^4 - 8}{e^8 - 1}$ and the optimal control is $u^*=\dfrac{10 e^4 - 4}{e^8 - 1}e^t$.

Below is the optimal control plot and the corresponding optimal trajectory. Optimal control Optimal trajectory

| cite | improve this answer | |
  • $\begingroup$ I don't understand how you integrated $\dot \lambda = \lambda$. $\endgroup$ – Rodrigo de Azevedo Jan 6 '18 at 10:18
  • $\begingroup$ @Rodrigo-de-Azevedo Indeed, I mistyped a "+" which later propagated through the calculation. Now it's corrected and coincides with your result :) $\endgroup$ – Dmitry Jan 6 '18 at 12:50

Integrating the differential equation $\dot x = -x + u$, we obtain

$$x(4) = e^{-4} \left( x(0) + \int_0^4 e^t \,u (t) \, \mathrm d t\right)$$

Since the fixed endpoints are $x(0) = 2$ and $x(4) = 5$, we obtain the following equality constraint

$$\int_0^4 e^t \,u (t) \, \mathrm d t = 5 e^4 - 2$$

Since the cost function is the energy of the input function $u$, we obtain the following constrained optimization problem in function $u : [0,4] \to \mathbb R$

$$\begin{array}{ll} \text{minimize} & \qquad\displaystyle\int_0^4 u^2 (t) \, \mathrm d t\\ \text{subject to} & \displaystyle\int_0^4 e^t \,u (t) \, \mathrm d t = 5 e^4 - 2\end{array}$$

Let the Lagrangian be

$$\mathcal L (t, u) := u^2 + \lambda e^t u$$

The Euler-Lagrange equation produces

$$u (t) = -\frac{\lambda}2 e^t$$

Using the equality constraint, we eventually obtain the Lagrange multiplier

$$\lambda = -4 \left(\frac{5 e^4 - 2}{e^8 - 1}\right)$$

and, thus, the optimal input function is

$$\bar u (t) := \color{blue}{2 \left(\frac{5 e^4 - 2}{e^8 - 1}\right) e^t}$$

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.