I am in the process of trying to solve an applied optimal control problem but I am not sure if my approach is correct. The problem is regarding a simple heating model between two rooms given by the system model $$\dot{\boldsymbol{x}} = \begin{bmatrix} -1.1 & 0.1 \\ 0.1 & -1.1 \end{bmatrix} \boldsymbol{x} + \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \boldsymbol{u}$$ and the room temperature starts at $\boldsymbol{x}(0)=[5,10]^T$ and the goal is to get the rooms close to the ambient temperature of $0$ by time $t_\text{f}=1$ using the least amount of energy. The cost function is given by

$$ J=\boldsymbol{x}^T(0)\boldsymbol{x}(0) + \int_0^1 \boldsymbol{u}^T\boldsymbol{u}~dt=x^2_1(0)+x^2_2(0) + \int_0^1 u_1^2(t)+u_2^2(t)~dt.$$ The control input $\boldsymbol{u}$ needs to be found that minimizes the cost function. Since the problem has a fixed time $t_\text{f}=1$ does that mean the problem is a "fixed time free state" optimization problem? Or is the state also fixed? Can this be solved with the Algebraic Ricatti Equation?

  • $\begingroup$ The Algebraic Ricatti Equation cannot be used in this case. $\endgroup$ – MachineLearner Apr 28 '19 at 17:37

The given problem can be stated as

$$\text{min: }J = S(x(t_\text{f}),t_\text{f})+\int_{t_0}^{t_\text{f}}\mathcal{L}(t,x(t),u(t))~dt$$ $$\text{subject to: } \dot{x}=f(t,x(t),u(t)), x(t_0)=x_0, u\in \mathcal{U}.$$

For your problem, the final time is fixed and the final state is free. You solve this problem by the following method.

  1. Step: Determine the Hamiltonian $$\mathcal{H}(t,x(t),u(t),\lambda(t))=\mathcal{L}(t,x(t),u(t))+\lambda^T(t)f(t,x(t),u(t))$$
  2. Step: Use $\min_{u\in \mathcal{U}} \mathcal{H}$ (for constrained inputs) or $\dfrac{\partial H}{\partial u}=0$ (for unconstrained inputs) to obtain $u^*(t)=u^*(t,x^*(t),\lambda^*(t))$
  3. Step: Plug $u^*(t)=u^*(t,x^*(t),\lambda^*(t))$ into the Hamiltonian to obtain $\mathcal{H}^*(t,x^*(t),\lambda^*(t))$
  4. Step: Solve the coupled ODE system

$$\dot{x}^*(t) = \dfrac{\partial \mathcal{H^*}}{\partial \lambda^*}$$ $$\dot{\lambda}^*(t) = -\dfrac{\partial \mathcal{H^*}}{\partial x^*}$$

with the initial condition $x*(t=t_0)=x_0$ and the final conditions $$\left[\mathcal{H}^*+\dfrac{\partial S}{\partial t} \right]_{t_\text{f}}\delta t_\text{f}+\left[\dfrac{\partial S^*}{\partial x^*} -\lambda^*\right]^T_{t_\text{f}}\delta x_\text{f}=0.$$ If the final time / final state is fixed we have $\delta t_\text{f}=0$ / $\delta x_\text{f}=0$.

  1. Step: Substitute $x^*(t)$ and $\lambda^*(t)$ from Step 2 to obtain the optimal control $u^*(t)$.

For the given problem we have.

  1. $\mathcal{H}=u_1^2+u_2^2+\lambda_1u_1+\lambda_2u_2-1.1x_1\lambda_1+0.1x_2\lambda_1+0.1x_1\lambda_2-1.1x_2\lambda_2$

  2. $$\dfrac{\partial \mathcal{H}}{\partial u}=\begin{bmatrix}2u_1+\lambda_1\\2u_2+\lambda_2 \end{bmatrix}=\begin{bmatrix}0\\0 \end{bmatrix}$$ $$\implies u^* = -[\lambda_1/2,\lambda_2/2]^T.$$

  3. $\mathcal{H}^*=\lambda_1^2/4+\lambda_2^2/4-\lambda_1^2/2-\lambda_2^2/2-1.1x_1\lambda_1+0.1x_2\lambda_1+0.1x_1\lambda_2-1.1x_2\lambda_2$

  4. $$\dfrac{d}{dt}\begin{bmatrix}x_1^*\\x_2^*\\\lambda_1^*\\\lambda_2^* \end{bmatrix} = \begin{bmatrix}-1.1 & 0.1 & -0.5 & 0.5\\0.1 & -1.1 & 0.0 & -0.5\\0.0 & 0.0 & -1.1 & 0.1\\ 0.0 & 0.0 & 0.1 & -1.1 \end{bmatrix} \begin{bmatrix}x_1^*\\x_2^*\\\lambda_1^*\\\lambda_2^* \end{bmatrix},$$ in which $\boldsymbol{x}^*(0)=[5,10]^T$ and $\boldsymbol{\lambda}^*(1)=2[x_1^*(1),x_2^*(1)]^T$.

  5. Solve the previous system and use the states to replace $\boldsymbol{\lambda}^*(t)$ in $\boldsymbol{u}^*(t)=-0.5\boldsymbol{\lambda}^*(t)$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.