Can anyone suggest me some references (papers, books, lecture notes) on the relation between GA and Information theory?

up vote 2 down vote accepted

There is definitely some general relation between the two subjects within complexity theory, and as likely one of the few people who know what both fields entail, I've not come across anything specific or even general linking the two fields in a direct manner. I've seen some research that ties the two fields separately to other areas in an indirect manner, but I doubt this is of any interest to you.

One of the few papers I'm directly aware of in the space is "Analaysis and improvement of genetic algorithms using concepts from information theory" by John Edward Milton, though it's been quite some time since I read it, and I recall it being a thesis.

The closest thing I can think to refer you to is some of Melanie Mitchell's work. She's written a book entitled "Complexity: A Guided Tour" (Oxford University Press, 2011) which touches on both subjects underneath the umbrella of complexity studies though in separate chapters. This book is best described as popular science and doesn't have many (any) equations at all, so it'll give you only an idea of their relationship. If you contact her directly via Twitter (I know she's on Twitter), perhaps she can give you more specifics.

I also maintain an open Mendeley group ITBio: Information Theory, Microbiology, Evolution, and Complexity which may be of some tangential help for your search. Unfortunately there are only a few dozen researchers who can function in the worlds of information theory and evolution/biology at the same time.

You might also delve into the work of Stuart Kauffman ("The Origins of Order" (Oxford University Press, 1993), though many serious information theorists refute the direction he takes mathematically as his grounding isn't in information theory itself) or Gregory Chaitin who have some interesting material tangential to the subject. The Santa Fe Institute may also have one or more interesting tangential researchers.

I'm quite interested to see/hear what your or others may come up with in this interesting space.


Subsequent addition: Knowing of your interest in this with relation to financial markets allows some additional references which you may want to look into as well. Just a few months ago researcher Nassim Nicholas Taleb wrote, "INFORMATION THEORY is the new central discipline." This isn't anything new to information theorists, but is interesting in relation to those in the areas of risk, uncertaintly, forecasting, finance, and economics.

First, for an overview, read the recent article Economic complexity: A different way to look at the economy by W. Brian Arthur after which you might want to take a look at any/many of Nassim Nicholas Taleb's books in his Incerto series, though I recommend starting with The Black Swan: Second Edition: The Impact of the Highly Improbable: With a new section: On Robustness and Fragility (and I would definitely recommend the supplemented second edition over the first). It's a non-mathematical best-seller outlining a new philosophical framework for viewing uncertainty in the financial markets. You could then move into some of his more mathematically motivated works, of which I would suggest that you may be able to download some of his unpublished work for free from his Facebook page.

Finally, you can explore the rich literature within economics and complexity including W. Brian Arthur's book Complexity and the Economy (Oxford University Press, 2014) which is mentioned in the above referenced article. The bibliography of this book is sure to be rich with a variety of additional material.

  • Great sources! My ultimate interest is on financial market. I strongly believe (but not yet verified) that information theory and evolution theory (and algorithm) are very potential and ideal candidates to better understand finance, that's why I want to learn about these 2, but I don't want to dive too much on biology. I've started to read your suggested book "Complexity" of Melanie Mitchell, things are much more clear now! – SiXUlm Jan 23 '15 at 9:07
  • SiXUlm, Glad you've found these useful references. I've added some additional material on finance/economics for your benefit as well. – Chris Aldrich Jan 24 '15 at 3:09
  • Another excellent reference that has entered into the space since this question was asked is Cesar Hidalgo's book Why Information Grows: The Evolution of Order, from Atoms to Economies [amzn.to/1VuFghS]. You'll likely find some of his theory in the book light on math, but he's got some additional papers that may add additional details. – Chris Aldrich Jun 1 '16 at 4:54

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.