# How to Interpret Time Scales in a Dynamic System

Here I have a question about time scales in dynamic systems - for reference you can look at a previous question that spurs this one:

Minimizing the cost of a path in a dynamic system

That question was about finding the minimum cost path from $0$ to $c$ on the real line, where cost is quadratic in the size of the step at each time $t$, and there was a drift towards 0 of $-P(t)$, $P(t)$ being your position at time $t$. The question is, what is the minimum cost path, when you minimize over size of step each period, and number of steps, subject only to the constraint of eventually getting to $c$.

The solution in this discrete time problem is to see that the problem is essentially linear, with linear constraints, and a solution easy to characterize for a given length of path $T$, and whose cost is decreasing in $T$, so that the minimum cost path is to take a infinite amount of time to get from o to $c$. Taking very very small steps is the way to go. Well and good, the question is answered.

It may then seem natural to ask, "What about the continuous time analog to that problem?" That is, if the original dynamics are given by

$$P(t) = (1-\gamma) P(t-1) + \gamma x(t)$$

where $\gamma<1$ limits fast I can move, and which can be written

$$\frac{P(t)- P(t-1)}{\gamma} = - P(t-1) +x(t),$$

What if we send $\gamma$ to zero, essentially discretizing the time interval finer and finer? The problem then becomes to solve the following problem: Minimize the cost:

$$\int _1^T ( x(t))^2 dt,$$

subject to the constraint that P(0) = 0, P(T) = c, and P goes from 0 to c, and $P$ follows the following dynamics:

$$\dot{P} = -P(t) + x(t)$$

The constraint is essentially that $P$ solves a particular ODE. I ask mathematica about it, and am informed that $P$ has the following form:

$$P(z) = e^{-z} ( K + \int_1^{z} e^t x(t) dt),$$

where $K$ is a constant of integration. So, adding in the boundary conditions, the constraint on the minimization problem is that

$$e^{-T} \int_1^{T} e^t x(t) dt = c,$$ that is, whatever $x(t)$ is, $P$ must end up at $c$ at time $T$. So the lagrangian is

$$L = \int _1^T ( x(t))^2 dt - \lambda (e^{-T} \int_1^{T} e^t x(t) dt - c)$$

But note, just as before, this lagrangian is simply quadratic in $x(t)$, and can be solved for a fixed $T$.

When I ask mathematica to do so, I find that

$$x(t) = \frac{e^t c}{e^{t-1} - e},$$ Which is very simple, linear in distance to be moved $c$. Note that this doesn't depend on $T$, the total time length of the path; indeed, when we minimize cost with respect to $T$, we find that cost in minimized for $$T = \frac{1}{2} (1 - 2 ProductLog[-1, -(1/(2 \sqrt{e}))]),$$ where "ProductLog(-1,z)" is mathematica's way of calculating Lambert's W function. So a finite $T$ is cost minimizing, and it is about 2.25643.

Now, this isn't necessarily a contradiction to the discrete case; after all, a finite length of time in the continuous model is an infinite number of periods in the discrete model, so the answers are not in conflict. However, the answer in the continuous model might easily have been "infinite amount of (continuous) time," but it wasn't. My questions are

1. Why not?
2. How to interpret the value of cost-minimizing $T$ - what does it mean? How much "time" is that?

The point of all this is to "justify" some simluations of the discrete system we are doing; we have a problem where we are simulating random shocks to a dynamic system, and we have some theorems that say, mean time to escape of from some point is a function of this cost minimization problem, IE the cost minimizing way to escape is the dominant way to escape, the most likely way to escape. But we are having a hard time interpreting the cost minimizing escape; it says that in the discrete case, the dominant escape should take an infinite amount of time, while in the continuous case(which the theorems are actually written for), it takes a finite amount of time, a time which seems to have nothing to do with the distance attempting to be escaped!

Well, obviously no one had anything to say on this issue - is it due to a poorly framed question? Would this question be more appropriate at mathoverflow?

-
Something's not quite right: the expression for $x(t)$ diverges at $t = 2$. Are you sure you had Mathematica solve the right thing? –  Rahul Sep 8 '11 at 6:24

This question keeps being bumped by the Community bot, so I guess I should try giving it a proper answer. $\newcommand{\d}{\mathrm d}$

You want to minimize $\int_0^T x(t)^2\ \d t$ with $\dot P(t) = -P(t) + x(t)$, subject to the constraints $P(0) = 0$ and $P(T) = c$. This is essentially a problem in the calculus of variations. Your Lagrangian, i.e. the integrand in the objective function, is $\mathcal L(t,P,\dot P) = x^2 = \big(P + \dot P\big)^2$. The optimal solution for $P(t)$ must satisfy the corresponding Euler-Lagrange equation, \begin{align} 0 &= \frac{\partial \mathcal L}{\partial P} - \frac{\d}{\d t}\frac{\partial \mathcal L}{\partial \dot P} \\ &= 2(P + \dot P) - \frac{\d}{\d t}\big(2(P + \dot P)\big) \\ &= 2(P - \ddot P). \end{align} Applying the given boundary conditions yields a unique solution to this differential equation, $$P(t) = a(e^t - e^{-t}),$$ where $a = c/(e^T - e^{-T})$. The corresponding $x(t)$ is $2ae^t$, and the total cost is $$\int_0^T x(t)^2\ \d t = 2a^2(e^{2T}-1) = \frac{2c^2}{1 - e^{-2T}}.$$ As in the discrete case, the cost decreases asymptotically with $T$.

In your derivation, you're fine up to $e^{-T} \int_0^{T} e^t x(t)\ \d t = c$; I suspect the problem comes in somewhere between where you define your Lagrangian $$L = \int _1^T x(t)^2\ \d t - \lambda \left(e^{-T} \int_1^{T} e^t x(t)\ \d t - c\right)$$ and where you get Mathematica to solve it. I'm not really sure what you did there, but if I make a heuristic and totally nonrigorous argument that $\dfrac{\partial L}{\partial x(t)} = 0$, I get $2x(t)\ \d t - \lambda e^{-T}e^t\ \d t = 0$, or $x(t) = \text{const}\cdot e^{t}$, consistent with my solution.

-

Although I don't understand the specifics of your problem, let me try to help you with a more generic argument in line with the title of your question.

In contrast to the idea of studying ODE's taking the time-one flow ("discrete-izing"), the idea of studying the "continuous time analog" is, "usually", not very fruitful despite how attractive it is.

To sort of exemplify my comment, there is the Collatz conjecture . I'm aware of some tentatives of considering the "continuous time analog", and some have radically different dynamics (but are relevant since provide other ways and equivalences to study the conjecture). Let's not go off topic, though: we're talking homeomorphisms.

Bottom line is that the dynamics of homeomorphisms is much richer that the dynamics of flows: the key feature (perhaps) is that orbits of (autonomous) flows cannot cross.

Now, this feels more like your problem: Perhaps the orbits minimizing costs in infinite steps were "crossing", in some sense, but you destroyed these solutions by making your problem smooth.

Again: take my intuition with a grain of salt, but I sincerely hope that this helped somehow.

-