Understanding Takens' Embedding theorem

I am having some trouble understanding Takens' embedding theorem, and was hoping that someone with greater knowledge could help out.

Formally, the theorem goes as follows:

Let $M$ be a compact manifold of dimension $m$. For pairs $(\phi,y)$, where $\phi : M \rightarrow M$ is a smooth diffeomorphism (an invertible function that maps one differentiable manifold to another such that both the function and its inverse are smooth) and $y : M \rightarrow \mathbb{R}$ a smooth function, it is a generic property that the $(2m+ 1)$--delay observation map $\Phi_{(\phi,y)}: M \rightarrow \mathbb{R}^{2m+1}$ given by $$\label{eq:mapping} \Phi_{(\phi,y)}(x) = \left(y(x),y \circ \phi (x),\ldots,y \circ \phi^{2m} (x) \right)$$ is an embedding; by smooth' we mean at least $C^2$.

Suppose that a measured time series $y(1),y(2),...,y(N)$ lies on a $D$-dimensional attractor of an $n$ th-order deterministic dynamical system. The starting point obtains an embedding from the recorded data. A convenient, though not unique, representation is achieved by using delay coordinates, for which a delay vector has the following form:

$$\mathbf{y}(k) = [y(k),y(k-\tau),\ldots,y(k - (d_\text{e}-1)\tau)]^{\mathsf{T}},$$

where $d_\text{e}$ is the embedding dimension and $τ$ is the delay time. Takens has shown that embeddings with $d > 2n$ will be faithful generically so that there is a smooth map $f:\mathbb{R}^{d_\text{e}} \mapsto \mathbb{R}$ such that

$$y(k+1) = f(\mathbf{y}(k))$$

for all integers $k$, and where the forecasting time $T$ and $\tau$ are also assumed to be integers.

My issues:

• The time-series lives on some $D$-dimensional attractor, so that would be equivalent to saying we are measuring some system and we record data of dimension $D$? I.e. imagine we are measuring some system of stock prices consisting of three different stocks, and we sample this price at every $\Delta t$, then $D=3$?

• An $n^{th}$ order deterministic dynamical system, means that it has $n$ degrees of freedom? I don't understand what $n$ (or $m$ in the theorem actually is)?

• So assuming e.g. $n=4$, then as long as my $d_\text{e}=9$ or more I can accurately map from that space back to the measured space (this is still without knowing what $n$ actually represents)?

Here's some Lorenz data that might aid explanations:

1 Answer

Practical meaning of the Takens Theorem using your example

The butterlfly-like structure traced out by the trajectories of the Lorenz system is the attractor of this dynamics. Its properties contain useful information about the dynamics, e.g., that it’s chaotic and how the “wings” interact. In a typical situation you do not have access to all dynamical variables ($x$, $y$, and $z$), but only to one time series, say $z$.

Takens theorem now states that you can obtain a structure that is topologically equivalent to your attractor by means of a delay embedding. It further gives an upper bound for the required dimension of this embedding. However, this is not so useful in reality, as you do not know the quantities going into this. However, this estimate is usually too high: For example, Lorenz attractor can be embedded with a three-dimensional delay embedding, while the Takens Theorem only guarantees that a seven-dimensional embedding suffices.

Clarification

I presume that at least some of your confusion stems from the following sentence from your second quote:

Takens has shown that embeddings with $d > 2n$ will be faithful generically

Were this written in analogy to your first quote, the relation would have to be $d>2D$. (Note that this is not incorrect though, since $D>n$.)

The equivalences between your first and second quote are as follows:

first quote | second quote
M   attractor
m   D
–   n
–   d_e
`

Your questions

An $n^{th}$ order deterministic dynamical system, means that it has $n$ degrees of freedom? I don't understand what $n$ (or $m$ in the theorem actually is)?

You are correct regarding $n$. However, $n$ is not equal to the $m$ from the theorem. The closest equivalent to $n$ in your first quote is the dimension of some $ℝ^n$ into which $M$ is embedded.

The time-series lives on some $D$-dimensional attractor, so that would be equivalent to saying we are measuring some system and we record data of dimension $D$?

No. The dimension of the attractor is a property of the dynamics. It is independent of your number of actually measured observables.

For example, a limit-cycle dynamics has a one-dimensional attractor, as you can identify positions on the attractor with one real number¹, namely the phase. A quasiperiodic dynamics that is a superposition of two periodic dynamics with incommensurable frequencies has a dimension of two, as you need two phases to identify a position on the attractor. In general, the attractor is some subset of a $D$-manifold ($M$ in the first quote), which in turn is embedded in the $n$-dimensional state space of the dynamics (hence $D<n$). For example, for your Lorenz system, the butterfly-shaped structure traced out by the trajectories is the attractor.

I.e. imagine we are measuring some system of stock prices consisting of three different stocks, and we sample this price at every $\Delta t$, then $D=3$?

No, at best we have $n=3$ and that’s if those three stock prices interact with nothing else. If you have other external factors to consider, this adds degrees of freedom and thus increases $n$.

So assuming e.g. $n=4$, then as long as my $d_\text{e}=9$ or more I can accurately map from that space back to the measured space […]?

I think you mean the right thing, but I wouldn’t use the term measured space for the phase space or attractor, as the entire point of the Takens embedding is that you reconstruct a phase space or attractor that you cannot measure due to practical constraints.

Also note that in this statement you can replace $n$ by $D$ (see above) or even the box-counting dimension $D_B$ of the attractor (Theorem of Sauer, Yorke, and Casdagli).

¹ assuming that the number is mapped to the position in a reasonable (i.e., piecewise smooth) fashion

• An exceptionally good answer, bravo to you, sir. Some follow-up questions if I may; 1) I must admit that I am still confused w.r.t. $m$ and $M$ in the original theorem. Perhaps referring to the Lorenz figures might help; (a) $M$ can be construed as the butterfly shape above and (b) it patently lives in $\mathbb{R}^3$ hence we can say that $m=3$ for this example? – Astrid May 3 '17 at 8:41
• @Astrid: a) Yes. b) The embedding in your graphics is not a delay embedding; it makes use of all dynamical variables $x, y,$ and $z$. While the Lorenz attractor can indeed be embedded with $m=3$, this is not exactly the same thing. In fact, the Lorenz attractor is a rather unlucky example for explaining the Takens theorem, as the upper bound given by Takens is very high. (Also see my edit.) – Wrzlprmft May 3 '17 at 8:51
• No, you are of course correct, naturally what I have shown is not the delay-embedding, please excuse my oversight. But suppose we did use e.g. the $x$ measurements from the Lorenz system to reconstruct its phase-space, and then found that $x(t)$, $x(t-\tau)$ and $x(t-2 \tau)$ are an adequate (or the 'best' w.r.t. mutual information scores etc) basis, then we are saying that $m$ is in fact 3? This would also then imply that $d>2m$ is really a bit of a useless statement since we do not know what $m$ is for real experimental systems (most of the time)? – Astrid May 3 '17 at 8:54
• @Astrid: But suppose we did use – Correct. If the picture above were the result of a delay embedding, then $m=3$ for this embedding. (Not withstanding the fact that everything is then again projected to the two dimensions of a screen.) — This would also then imply that $d>2m$ is really a bit of a useless statement – Yes, see my first paragraph. The main practical result of the Takens theorem is that delay embeddings can be, well, embeddings (not for which dimension this happens). – Wrzlprmft May 3 '17 at 9:00
• You mean for Lorenz, m = 3, and 2m+1 = 7, so 6 previous time step is necessary for the whole picture. However, I still don't think the dimension of the Lorenz attractor is 3. What you mean by 3 is indeed the n = 3. – ArtificiallyIntelligence Nov 23 '18 at 13:58