Takens’ Embedding Theorem

Statement

Takens’ theorem is the 1981 delay embedding theorem of Floris Takens. It provides the conditions under which a smooth attractor can be reconstructed from the observations made with a generic function. Later results replaced the smooth attractor with a set of arbitrary box counting dimension and the class of generic functions with other classes of functions.

A delay embedding theorem uses an observation function to construct the embedding function. An observation function α must be twice-differentiable and associate a real number to any point of the attractor A. It must also be typical, so its derivative is of full rank and has no special symmetries in its components. The delay embedding theorem states that the

ϕ T ( x ) = ( α ( x ) , α ( f ( x ) ) , … , α ( f k − 1 ( x ) ) )

is an embedding of the strange attractor A.

Takens’ innovation was not in the general idea, which was already a widely accepted folk-theorem (at least at Warwick), building on work going back to at least the 50s, but in working out the details, particularly the ‘conditions’, and in providing a sound proof. (This is typical of how mathematics progresses.)

Application

Takens provides some formulae to be applied to experimental data. He says:

If, in the calculation of D(L+(p)), the limit would have the tendency of going to infinity, this would imply that representing the evolution on a finite dimensional manifold is a mistake. If on the other hand this limit would go to a non-integer, this would be evidence in favour of a strange attractor. …

If the experimental data do not clearly indicate the limits in the calculation of D(L+(p)) and H(L+(P)) to exist and to be finite, then both the Landau-Lifschitz and the Ruelle-Takens picture [as widely used in putative ‘applications’] are to be rejected as explanation of the experimental data.

For me, an important consequence of the conventional ‘picture’ is that they entail the existence of objective probability distributions, and hence the validity of much contemporary ‘probablism’, ‘science’ and ‘big data’.

The limit problem

In mathematics, the limit of a sequence is the value that the terms of a sequence “tend to”.[1] If such a limit exists, the sequence is called convergent. A sequence which does not converge is said to be divergent.[2] The limit of a sequence is said to be the fundamental notion on which the whole of analysis ultimately rests.[

In working on related problems to Takens at about the same time and since, I concur with the old folk wisdom that not only is this the ‘fundamental notion’ but that showing the existence of a limit is often the ‘heavy lifting’.

Experimentally, one can never be sure that one has a limit, especially in evolutionary systems that can display ‘punctuated equilibria’. Even in an engineered construct, such as an engine working in a laboratory, a strict notion of a limit would imply that the construct would last forever, even longer than the universe. So one can, I think, never apply Takens’ theory directly. One can take an engineered construct and apply the theory to the idealised construct: ‘as if’ it were going to continue as a fixed system forever. The results will then hold unless and until the system changes (perhaps it will run out of fuel or wear out). This is ‘common sense’ and – perhaps – doesn’t need to be said. More seriously, there is the problem of knowing what the dimension of the system is. One can put a reasonable upper bound on it by studying the design, but in practice engineered constructs can have unintended ’emergent behaviours’ (such as hunting in engine or wobbling in bridges). These increase the relevant dimension. So knowing when a limit has been reached is not a trivial problem, even for engineered constructs. But for engineered constructs one can at least reach conclusions about the idealised system ‘working as intended’.

For systems that are not entirely engineered (such as bridges being used by people) the problem of establishing limits would seem to depend on characterising the system as a whole, including the non-engineered bit. For example, the relevant characteristics of a bridge system might vary with climate change, or with what type of music was in vogue.

Working with engineers and on my own account, the notion of a short-term limit seems useful. Instead of trying to characterise the system as a whole, one simply seeks to characterise the behaviour as evidenced by the available data, without prejudging how representative that data is. A typical heuristic is to guesstimate an upper bound to the dimension, analyse the system with an upper bound increased by two, and see if it confirms your lower estimate. (I.e, allowing for two more dimensions makes no difference to the conclusion. The situation is similar to that for curve fitting.) If the extra dimensions do make a difference, then increase the dimensional allowance and repeat, making sure that the data is enough to avoid spurious results (as in curve fitting). If you succeed, you have reasonable grounds for saying that the real system has been being ‘like’ your model system, and (normally) some grounds for thinking that this will continue to hold for a while. But this analysis on its own cannot rule out the possibility of as yet unknown emergent properties, particularly when looking ahead over a greater length of time than that for which you have data. Thus if you want to rely on the real system continuing to approximate to the model, you will need to keep collecting and analysing data. This used to ‘go without saying’.

Implications and Impact

The general idea had long been recognized as critical to the practical application of ideas from dynamical systems theory. We rely on ‘the law of physics’ not because they have been ‘proven’, but that they continue to be widely ‘proven’ in the sense that, for the most part, experimenters and innovators are highly motivated and often able to disprove (or at least improve) them, and repeatedly try. But this is not the case for all areas claiming to be ‘scientific’.

A web search reveals many papers which claim to detail the implications, but which often omit some or all of Takens’ conditions. Thus the mathematics does seem to have stimulate a lot of pseudo-applications, disregarding the conditions. It is, perhaps, understandable that some people blame ‘sophisticated mathematics’ for some of our ills, but given Takens’ own warnings (which the wider mathematical community at the time well understood) and what seems to me to be widespread engineering ‘good practice’ (at least in the UK), I think this unfair. I would rather blame people who, for whatever reason, in whatever field, act ‘pragmatically’ as if the negative aspect of Takens’ theorem could be ignored. More importantly, we need to improve practice, recognizing when Takens’ theorem invalidates much that is relied on in pseudo-sciences and elsewhere. For example, when a client asks a statistician for a ‘probability’, who is responsibility for considering the stability of the system under consideration. Should one supply the probability ‘looking backwards’, as derived from the data, or be actively looking for signs that the system might be about to undergo a radical change?

P.S. My own view is that there can never be any grounds for certainty that any real system ‘is’ technically within the scope of Takens’ theorem, but we are surely justified in using its cautionary aspect to critique the views of those who act ‘as if’ all the world’s (nothing but) a dynamical system.