# What is a “recurrent model” in forecasting

In this book, there is a chapter titled Recurrent Models (you can see that chapter in Google books) but it's very short and some parts are not very clear to me. Recurrent Models seem to refer to a regression of the following form:

$$y_{k} = f(y_{k-m},..., y_{k-1},x_{k-m},..., x_{k-1}) \hspace{20mm}\text{for} \; k = m+1,....,n$$

However, I would like to get more information about this topic but maybe the name recurrent model is not a very common one, so I'm having a hard time getting more notes or books about it. Can anyone give me some pointers?

Thanks

-
May refer to Recurrent Neural Network models (Googling recurrent model yields much material related to RNNs). RNNs are dynamical neural network models where there is feedback, as opposed to the output being a static function of the input. .. I would stay away from this stuff and learn algebra and topology and metric spaces instead. –  alancalvitti Feb 14 '13 at 4:36
I don't really think it refers to RNN (even if RNN can also be used with regression) but just out of curiosity, why did you suggest to stay away from RNNs? –  Robert Smith Feb 14 '13 at 4:49
Was going by the top Google hits. To give you an example of how excruciating NNs can be, my thesis advisor analyzed a small network of less than 10 neurons (Hopfield type, but w/o symmetries) coupled to an external environment, resulting in 30 pages of (Mathematica assisted) bifurcation analysis to understand them. –  alancalvitti Feb 14 '13 at 4:55
Now I understand your suggestion :-) –  Robert Smith Feb 14 '13 at 5:03

## 1 Answer

I don't know what you need this for, but what this author calls 'recurrent' seems a general version of autoregressive (AR) or vector-autoregressive (VAR) models. While these, to my knowledge, are assumed linear in their evolution over time, I'm not sure if the more general functional relationship of this book's definition is actionable in practice. If you need to estimate something along these lines, you might want to start reading about these well-understood and investigated models. Here's the wiki link: http://en.wikipedia.org/wiki/Vector_autoregression. A good book covering this in great detail is Hamilton, Time Series analysis.

Edit (see comment below): I touched base with my gf. While not precisely under the name "recurrent models," she suggested to google "Real-time computing without stable states," and the researcher names Maass, Buonomano, and Jaeger; also a paper by Abbott&Sussillo on recurrent nets for computation. This is how 'recurrent' models are used in contemporary theoretical neuroscience, which may or may not be what you are interested in. There should be an introductory article on Scholarpedia, eg, http://www.scholarpedia.org/article/Echo_state_network.

-
Thank you for your answer. I don't need this for a particular purpose. I have already read about AR models so 'recurrent models' indeed seem like generalized AR models, but a quick google search didn't return something similar to what I'm looking for. Hamilton's book covers AR models too? By the way, the author more or less motivates this using fine-state machines, so autoregressive and recurrent models are approximations to Moore and Mealy machines, respectively. –  Robert Smith Feb 14 '13 at 4:42
Hamilton was encyclopedically complete when I was in grad school (~late '90s) for anything (V)AR under the unifying theme of GMM; I don't know if it got updated after. It's focused heavily towards applications in economics/finance, so might not be best for neuroscience and FSM applications as neuroscience at least simulates heavily non-linear information transmission (my gf is a neuro grad student currently). So I don't know. :) –  gnometorule Feb 14 '13 at 13:44
@Robert Smith: in any case, it's good to see The Cure passionate about math...:) –  gnometorule Feb 14 '13 at 13:45
@RobertSmith: see an edit I added above, which maybe is closer to what you are interested in. –  gnometorule Feb 14 '13 at 18:05
Thanks! It seems that echo state networks are similar to recurrent models but the motivation is quite different. By the way, I read some parts of Hamilton's book and it covers AR and VAR models but it's very hard to know whether it contains something along the lines of recurrent models under a different name. Anyway, 1+ for the effort and for the reference to The Cure. –  Robert Smith Feb 15 '13 at 1:57