Time Series Analysis and Prediction: Andreas S. Weigend

The desire to predict the future and understand the past drives the search for laws
that explain the behavior of observed phenomena; examples range from the irregularity in a heartbeat to the volatility of a currency exchange rate. If there
are known underlying deterministic equations, in principle they can be solved to
forecast the outcome of an experiment based on knowledge of the initial conditions. To make a forecast if the equations are not known, one must find both the
rules and the state of the system. This chapter focuses on phenomena for which
underlying equations are not given; the rules that govern the evolution must be
inferred from regularities in the past. For example, the motion of a pendulum
or the rhythm of the seasons carry within them the potential for predicting their
future behavior from knowledge of their oscillations without requiring insight into
the underlying mechanism. We will use the terms "understanding" and "learning"
to refer to two complementary approaches taken to analyze an unfamiliar time
series. Understanding is based on explicit mathematical insight into how systems
behave, and learning is based on algorithms that can emulate the structure in a time
series. More specifically, we use information theory to obtain some insights into
the system, and we use neural networks to build models that emulate the system’s
behavior. In both cases, the goal is to explain observations; the important related

problem of using knowledge about a system for controlling it in order to produce
some desired behavior is discussed in Chapter 11 in this volume by Narendra
and Li.