I am trying to minimize an time-integral of a linear function with respect to differential equations. The problem is formally defined as follows:

Given $\lambda< \mu_1, \mu_2$ fixed parameters(thus $x(s)$ will hit 0 at some point in time and stay there) and $c_1, c_2$ time independent, fixed cost coefficients :

$v(x_1, x_2) = \min_{u,p} \int\limits_T \ [c_1 X_1(s) + c_2 X_2(s)]ds$ \

subject to

$\dot{X_1}(s) = p(s)\lambda - u(s)\mu_1$;

$\dot{X_2}(s) = (1- p(s))\lambda - (1-u(s))\mu_2$;

$ X_i(s) = x_i$

$0 \leq u(s) \leq 1$

$0 \leq p(s) \leq 1$

Thus $u(s), p(s)$ are the controls or decision variables. What would be the methodology to follow in this case?

If I understood properly, for this system to reach optimality HJB equation must be satisfied :

$\min_{u,p} (p(s)\lambda - u(s)\mu_1)\frac{\partial v}{\partial x1} + (1- p(s))\lambda - (1-u(s))\frac{\partial v}{\partial x2} + c_1 x_1 + c_2 x_2 = 0 $

But without knowing the exact form of $v$ I can not check whether the equation above holds or not. I guess one way to progress is to guess optimal $v$, but then again how such an argument would work without being circular.

Any help, references, comments appreciated... Thanks in advance..

  • $\begingroup$ Maybe you should not start with HJB equation but with Pontryagin's min principle. But let me point out here that your ode can be integrated with respect to s such that $X_{1}$ and $X_{2}$ only depend on $p$ and $u$ and some constants. Put this into your objective and use first order optimality conditions to obtain a system, where you only have box constraints...(that is $u$ and $p$ are bounded)... Does this help? $\endgroup$ – Alex May 12 '13 at 20:04

Use PMP (Pontryagen maximum principle). Pontryagen function (hamiltonian) in your problem is

$$ H = (c_1x_1+c_2x_2) + \phi_1(\lambda p + \mu_1 u) + p_2(\lambda (1-p) + \mu_2 (1-q)) $$

Here $\phi_1$ and $\phi_2$ are the adjoint variables to $x_1$ and $x_2$. So

$$ \dot\phi_i = -\frac{\partial}{\partial x_i}H $$

$$ \dot\phi_1 = c_1\ \ \mbox{ and }\ \ \dot \phi_2=c_2 $$

and control variables $u$ and $p$ can be found from maximum principle:

$$ H\to\max_{u,p} $$

i.e. $p=sign(\phi_1-\phi_2)$ and $u=sing(\mu_1\phi_1 - \mu_2\phi_2)$.As a result you will have simple ODE system with 4 variables $x_1$, $x_2$, $\phi_1$ and $\phi_2$. It can be solved easely by standard methods

  • $\begingroup$ thanks I have already solved it that way! $\endgroup$ – Roark May 21 '13 at 21:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.