# Applied Optimal Control - Minimize Integral Cost Function

I am in the process of trying to solve an applied optimal control problem but I am not sure if my approach is correct. The problem is regarding a simple heating model between two rooms given by the system model $$\dot{\boldsymbol{x}} = \begin{bmatrix} -1.1 & 0.1 \\ 0.1 & -1.1 \end{bmatrix} \boldsymbol{x} + \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \boldsymbol{u}$$ and the room temperature starts at $$\boldsymbol{x}(0)=[5,10]^T$$ and the goal is to get the rooms close to the ambient temperature of $$0$$ by time $$t_\text{f}=1$$ using the least amount of energy. The cost function is given by

$$J=\boldsymbol{x}^T(0)\boldsymbol{x}(0) + \int_0^1 \boldsymbol{u}^T\boldsymbol{u}~dt=x^2_1(0)+x^2_2(0) + \int_0^1 u_1^2(t)+u_2^2(t)~dt.$$ The control input $$\boldsymbol{u}$$ needs to be found that minimizes the cost function. Since the problem has a fixed time $$t_\text{f}=1$$ does that mean the problem is a "fixed time free state" optimization problem? Or is the state also fixed? Can this be solved with the Algebraic Ricatti Equation?

• The Algebraic Ricatti Equation cannot be used in this case. – MachineLearner Apr 28 '19 at 17:37

The given problem can be stated as

$$\text{min: }J = S(x(t_\text{f}),t_\text{f})+\int_{t_0}^{t_\text{f}}\mathcal{L}(t,x(t),u(t))~dt$$ $$\text{subject to: } \dot{x}=f(t,x(t),u(t)), x(t_0)=x_0, u\in \mathcal{U}.$$

For your problem, the final time is fixed and the final state is free. You solve this problem by the following method.

1. Step: Determine the Hamiltonian $$\mathcal{H}(t,x(t),u(t),\lambda(t))=\mathcal{L}(t,x(t),u(t))+\lambda^T(t)f(t,x(t),u(t))$$
2. Step: Use $$\min_{u\in \mathcal{U}} \mathcal{H}$$ (for constrained inputs) or $$\dfrac{\partial H}{\partial u}=0$$ (for unconstrained inputs) to obtain $$u^*(t)=u^*(t,x^*(t),\lambda^*(t))$$
3. Step: Plug $$u^*(t)=u^*(t,x^*(t),\lambda^*(t))$$ into the Hamiltonian to obtain $$\mathcal{H}^*(t,x^*(t),\lambda^*(t))$$
4. Step: Solve the coupled ODE system

$$\dot{x}^*(t) = \dfrac{\partial \mathcal{H^*}}{\partial \lambda^*}$$ $$\dot{\lambda}^*(t) = -\dfrac{\partial \mathcal{H^*}}{\partial x^*}$$

with the initial condition $$x*(t=t_0)=x_0$$ and the final conditions $$\left[\mathcal{H}^*+\dfrac{\partial S}{\partial t} \right]_{t_\text{f}}\delta t_\text{f}+\left[\dfrac{\partial S^*}{\partial x^*} -\lambda^*\right]^T_{t_\text{f}}\delta x_\text{f}=0.$$ If the final time / final state is fixed we have $$\delta t_\text{f}=0$$ / $$\delta x_\text{f}=0$$.

1. Step: Substitute $$x^*(t)$$ and $$\lambda^*(t)$$ from Step 2 to obtain the optimal control $$u^*(t)$$.

For the given problem we have.

1. $$\mathcal{H}=u_1^2+u_2^2+\lambda_1u_1+\lambda_2u_2-1.1x_1\lambda_1+0.1x_2\lambda_1+0.1x_1\lambda_2-1.1x_2\lambda_2$$

2. $$\dfrac{\partial \mathcal{H}}{\partial u}=\begin{bmatrix}2u_1+\lambda_1\\2u_2+\lambda_2 \end{bmatrix}=\begin{bmatrix}0\\0 \end{bmatrix}$$ $$\implies u^* = -[\lambda_1/2,\lambda_2/2]^T.$$

3. $$\mathcal{H}^*=\lambda_1^2/4+\lambda_2^2/4-\lambda_1^2/2-\lambda_2^2/2-1.1x_1\lambda_1+0.1x_2\lambda_1+0.1x_1\lambda_2-1.1x_2\lambda_2$$

4. $$\dfrac{d}{dt}\begin{bmatrix}x_1^*\\x_2^*\\\lambda_1^*\\\lambda_2^* \end{bmatrix} = \begin{bmatrix}-1.1 & 0.1 & -0.5 & 0.5\\0.1 & -1.1 & 0.0 & -0.5\\0.0 & 0.0 & -1.1 & 0.1\\ 0.0 & 0.0 & 0.1 & -1.1 \end{bmatrix} \begin{bmatrix}x_1^*\\x_2^*\\\lambda_1^*\\\lambda_2^* \end{bmatrix},$$ in which $$\boldsymbol{x}^*(0)=[5,10]^T$$ and $$\boldsymbol{\lambda}^*(1)=2[x_1^*(1),x_2^*(1)]^T$$.

5. Solve the previous system and use the states to replace $$\boldsymbol{\lambda}^*(t)$$ in $$\boldsymbol{u}^*(t)=-0.5\boldsymbol{\lambda}^*(t)$$