I noticed that the most simple numerical approximation of a higher order-differential equation has the same form as the numerical approximation of a delayed first-order differential equation. This leads me to the following hypothesis:

Hypothesis: Delayed first-order differential equations can be approximated by higher-order ordinary differential equations.

I wanted to investigate to what extent this is true so I came up with a method to find approximations of delayed differential equations. However, my initial tests seem to indicate that this approach doesn't work. I hope someone can explain why, and if there is a better, similar approach.

Here is my approach:

Take the simple delayed first-order differential equation for arbitrary function $f$: $$\dot x(t+1)=f(x(t))$$ By taking $\Delta x =1$, this is numerically approximated by the difference equation: $$x_{t+2}-x_{t+1}=f(x_t)$$ adding to both sides $x_t-x_{t+1}$, gives: $$x_{t+2}-2x_{t+1}+x_t=f(x_t) - (x_{t+1}-x_t)$$ Which, if we again take $\Delta x =1$, is the numerical approximation of $$\ddot x (t)=f(x(t))-\dot x(t)$$

Hence we might approximate a simple first-order delayed equation by a higher order non-delayed equation: $$\dot x(t+1)=f(x(t)), \quad \text {is approximated by}\quad \ddot x(t)+\dot x(t)=f(x(t)) $$ By the same approach we could show that $$\dot x(t+2)=f(x(t)), \quad \text {is approximated by}\quad \dddot x(t)+2\ddot x(t) +\dot x(t)=f(x(t))$$ $$\dot x(t+3)=f(x(t)), \quad \text {is approximated by}\quad \ddddot x(t)+3\dddot x(t) -3\ddot x(t)+\dot x(t)=f(x(t))$$ And so forth...

However, I've been using $\Delta x =1$ here. If we take $\Delta x=1/n$, and gradually increment $n$ upwards, a similar pattern to the one for larger delays occurs: $$\dot x(t+1)=f(x(t))$$ is approximated for $n=2$ by: $$2\cdot (x(t+1+\frac {1}{2})-x(t+1))=f(x(t))$$ using the same approach as above, but $\Delta x = \frac{1}{2}$ instead of $\Delta x = 1$, and a lot of tedious algebra, one can show that this is equivalent to the approximation of
$$2^{-2}\dddot x(t)+2^{-1}\cdot 2 \ddot x(t)+2^{-0}\dot x(t)$$

Hence we might approach a closer and closer approximation of $\dot x(t+1)=f(x(t))$, by taking increasingly larger $n$, and smaller $\Delta x=\frac{1}{n}$, as follows:

using $n=2$ (resulting in $\Delta x=\frac {1}{2} $): $$\dot x(t+1)=f(x(t)), \quad \text {is approximated by}\quad n^{-n}\cdot \dddot x(t)+n^{-n+1}\cdot 2 \ddot x(t)+n^{-n+2} \cdot \dot x(t)$$ and using $n=3$: $$\dot x(t+1)=f(x(t)), \quad \text {is approximated by}\quad $$ $$n^{-n}\cdot \ddddot x(t) +n^{-n+1}\cdot 3\dddot x(t)+n^{-n+2} \cdot 3\ddot x(t)+ n^{-n+3}\cdot \dot x(t)$$

And so forth...

The form of this approximation is the same as for $\Delta x =1$, but with higher delays, except for the coefficients of $n$.

The most important point I noticed about this approximation is that as we increase $n$, increasingly higher order derivatives are added, but the coefficients of those derivatives seem to decrease hyperexponentially (i.e. $n^{-n}$). This made me hope that perhaps the approximations would converge quickly to the delayed equation as $\Delta x$ decreases.

However, I did some tests on the delayed equation $\dot x(t)=x(t-1)$, and $\dot x(t) = x^2(t-1)$, where my approach fails miserably. increasing $n$ actually deteriorates the approximation.

On the one hand (in hindsight) this makes sense to me, since increasing the order of the derivative should increase the rate of growth of $x$ for large $t$, but it still bugs me that it doesn't work, despite the fact that the difference equations for the two are so similar.

So my question is: Why doesn't my approach work, given that the difference-equation approximations of delayed-differential-equations, and higher-order-differential equations have the same form? and more importantly, is there a different but similar approach to approximate delayed-differential-equations?

  • $\begingroup$ A delay system can sometimes be reduced to a higher ODE system, thus I think it can also be approximated by a higher ODE system. But I am not an expert in the computational aspect of your approach. Sound great though, good luck! :) $\endgroup$
    – Paichu
    Oct 22, 2016 at 4:09

2 Answers 2


If looking for a solution for $\dot x(t+1)=f(x(t))$ on $[0,\infty]$, take any continuous "seed" function $x:[-1,0]\to\Bbb R$ with $x(0)=x_0$ and define the solution as $$ x(t)=x_0+\int_0^t f(x(s-1))ds $$ The values on $[0,1]$ are well-defined using the values on $[-1,0]$, the values on $[1,2]$ by the values on $[0,1]$ and consequently $[-1,1]$ etc. By construction this function is continuous and by the integration it is differentiable on $(0,\infty)$.

As you can see, you get lots of solutions that satisfy this delay differential equation. Before starting to approximate, you will have to define which of them exactly you want to approximate.

  • $\begingroup$ I know about this approach. I'd like to see if it's possible to do an approximation of a delay-differential equation using a non-delay differential equation. I suggested to do it using higher-order differential equations, since the difference equations look a like. The reason I'm interested in this is because it would give me insight into the nature of delay-differential equations. $\endgroup$
    – user56834
    Oct 15, 2016 at 17:32
  • $\begingroup$ to answer your point: "you get lots of solutions that satisfy this delay differential equation. Before starting to approximate, you will have to define which of them exactly you want to approximate.", I assumed that the initial seed function could be incorporated as initial conditions in the ordinary differential equations. ofcourse it would highly restrict the possible initial conditions, compared to that of a delay differential equation, but I assumed there could be a good approximation of different seed functions this way. $\endgroup$
    – user56834
    Oct 15, 2016 at 17:41

A delay can be written as a differential $e^Dx(t)=x(t+1)$. So it might make sense to approximate $$\dot{x}(t+1)=f(x(t))$$ by cutting off the following series at some $n$, $$e^DDx(t)=f(x(t))$$ $$(\cdots+\frac{D^n}{(n-1)!}+\cdots+D^2+D)x(t)=f(x(t))$$ This ties in with Lutz Lehmann's answer in that an infinite number of initial data is required to specify the solution.

For example, approximate $\dot{x}(t+1)=f(x)$ by $(\frac{D^5}{4!}+\cdots+D^2+D)x(t)=f\circ x(t)$.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .