Consider a continuous-time optimal control problem where we store all available information up to time $t$ as our state ($S_t=\mathcal{I}_t$). At each time $t$, there are possibilities for updating the state. 1) A new piece of information arrives at time $t$ and we should add $\{(t,a_t)\}$ to $\mathcal{I}_t$, 2) A currently stored pice of information should be removed from $\mathcal{I}_t$, that is, a piece of information like ${(s,a_s)} $ is removed from $\mathcal{I}_t$; 3) or there is no change.

In discrete-time, it is possible to write $\mathcal{I}_{t+1}$ based on $\mathcal{I}_{t}$ and three possibilities. However, I am wondering how to write it in continuous time.

Since there is no rate for arrival of new information or removal of a piece of information, I am confused about how to write it.

$\mathcal{I}_{t+\delta t}= \mathcal{I}_{t} \cup \{(t,a_t)\} \text{ in case of a new arrival in } [t,t+\delta t]$?

I would be thankful if you can share a reference that has used the same modeling assumptions.

Edited: More description is added.

Suppose people enter a bank through a Poisson process and the time that is spent by each customer follows a known probability distribution. I need to store all information related to people who are still in the bank at each time. In this system, the state of the problem evolves when a new customer enters or leaves the bank and it is not possible to write the dynamic of the problem as a differential equation. Moreover, the state of the problem is a set of tuples.

  • $\begingroup$ Why do you want to write it in continuous time? $\endgroup$ Oct 5, 2021 at 7:21
  • $\begingroup$ The nature of my problem is continuous-time and I am wondering how to write the updating formula for states. $\endgroup$
    – Amin
    Oct 5, 2021 at 15:42
  • $\begingroup$ Maybe a hybrid formulation, using a "flow" and "jump" description, might be a better fit? $\endgroup$ Oct 5, 2021 at 17:04


You must log in to answer this question.

Browse other questions tagged .