I am proficient in standard dynamic programming techniques. In the standard textbook reference, the state variable and the control variable are separate entities. However, I have seen examples in economics, in which a single variable, let's say consumption, is both a state variable and a control variable simultaneously.

This is very strange. Can the same variable be a control variable and state variable simultaneously? Is it allowed in Bellman equation?

  • $\begingroup$ Please state if using the usual textbook Bellman equation formulation as well as its derivatives would yield the correct, or incorrect, solution. $\endgroup$ – he wei Oct 23 '15 at 9:05

If a state variable $x_t$ is the control variable $u_t$, then you can set your state variable directly by your control variable since $x_t = u_t$ ($t \in {\mathbb R}_+$).

However, this problem would not a dynamic control problem any more, as there are no dynamics. It becomes a static optimization problem. For example, if the objective function is $J = \int_{0}^{t_f} f(x_t, t) dt$, then $\forall t_1,t_2$ $f(x_{t_1}, t_1)$ and $f(x_{t_2}, t_2)$ are independent. In this case, you can optimize $f(x_t, t)$ for each $t$, and the curve $x_t$ is the answer for your optimal control law.

  • $\begingroup$ Even if it is a static problem. Is it OK to use the standard textbook formulations on Bellman equation as well as its derivatives to solve the problem. Would this generate correct results? $\endgroup$ – he wei Oct 23 '15 at 7:26
  • $\begingroup$ I think HJB works, since it is derived from the principle of optimality. However, please let the item containing the system function be zero, since the system state is independent of the system function but indeed dependent on your control input. As a result, just optimize $f(x_t, t)$ or $f(x_t,t)$ is enough to derive the optimal $u_t$ first, since the system function item disappears. Still, you can solve this HJB equation (it becomes the ODE rather than PDE), based on the derived control law $u_t$. All above is my idea, think it could be helpful. You can also wait for other better answers :-) $\endgroup$ – Ryan Oct 23 '15 at 9:48
  • $\begingroup$ Thanks Ryan. Hope there is more complete answer. $\endgroup$ – he wei Oct 23 '15 at 11:35
  • $\begingroup$ @hewei You are very welcome :-) $\endgroup$ – Ryan Oct 23 '15 at 12:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.