minimization problem on differential equations - optimal control

I am trying to minimize an time-integral of a linear function with respect to differential equations. The problem is formally defined as follows:

Given $\lambda< \mu_1, \mu_2$ fixed parameters(thus $x(s)$ will hit 0 at some point in time and stay there) and $c_1, c_2$ time independent, fixed cost coefficients :

$v(x_1, x_2) = \min_{u,p} \int\limits_T \ [c_1 X_1(s) + c_2 X_2(s)]ds$ \

subject to

$\dot{X_1}(s) = p(s)\lambda - u(s)\mu_1$;

$\dot{X_2}(s) = (1- p(s))\lambda - (1-u(s))\mu_2$;

$X_i(s) = x_i$

$0 \leq u(s) \leq 1$

$0 \leq p(s) \leq 1$

Thus $u(s), p(s)$ are the controls or decision variables. What would be the methodology to follow in this case?

If I understood properly, for this system to reach optimality HJB equation must be satisfied :

$\min_{u,p} (p(s)\lambda - u(s)\mu_1)\frac{\partial v}{\partial x1} + (1- p(s))\lambda - (1-u(s))\frac{\partial v}{\partial x2} + c_1 x_1 + c_2 x_2 = 0$

But without knowing the exact form of $v$ I can not check whether the equation above holds or not. I guess one way to progress is to guess optimal $v$, but then again how such an argument would work without being circular.

• Maybe you should not start with HJB equation but with Pontryagin's min principle. But let me point out here that your ode can be integrated with respect to s such that $X_{1}$ and $X_{2}$ only depend on $p$ and $u$ and some constants. Put this into your objective and use first order optimality conditions to obtain a system, where you only have box constraints...(that is $u$ and $p$ are bounded)... Does this help? – Alex May 12 '13 at 20:04

Use PMP (Pontryagen maximum principle). Pontryagen function (hamiltonian) in your problem is

$$H = (c_1x_1+c_2x_2) + \phi_1(\lambda p + \mu_1 u) + p_2(\lambda (1-p) + \mu_2 (1-q))$$

Here $\phi_1$ and $\phi_2$ are the adjoint variables to $x_1$ and $x_2$. So

$$\dot\phi_i = -\frac{\partial}{\partial x_i}H$$

$$\dot\phi_1 = c_1\ \ \mbox{ and }\ \ \dot \phi_2=c_2$$

and control variables $u$ and $p$ can be found from maximum principle:

$$H\to\max_{u,p}$$

i.e. $p=sign(\phi_1-\phi_2)$ and $u=sing(\mu_1\phi_1 - \mu_2\phi_2)$.As a result you will have simple ODE system with 4 variables $x_1$, $x_2$, $\phi_1$ and $\phi_2$. It can be solved easely by standard methods

• thanks I have already solved it that way! – Roark May 21 '13 at 21:52