While reading the book "Algorithms to live by" by Brian Christian and I came across the concept of probability distributions with memory. In the text they mention the power law distribution as an example.

Doing some online search I did not find much information about the topic:

  • what is the definition of a probability distribution with memory?
  • Why a power law distribution is a prob. distribution with memory?
  • or, what are the basic properties of these distributions...

I only found some non very generic papers on the topic.

I wonder if anyone has knowledge about this or if someone could recommend some useful and accessible literature to a non-expert.


Usually, when someone talks of "long memory" typically they are referring to a a time series process with an unbounded spectrum as frequency goes to zero and is called a “long memory process.” A class of processes with this property are the “fractionally integrated” series generated by

$$(1-B)^dX_t = Y_t,$$

with $B$ the lag operator and $Y_t$ is a stationary series and $X_t$ as a fractionally integrated series.

For these types of time series,

$$E[X_t] \sim m t^d,$$

and the autocorrelation function

$$\rho(k) \sim c k ^{2d-1}$$

possesses "long memory" because the autocorrelations DO NOT decay exponentially (like typical processes do when their variance < $\infty$) - they decay via a power-law and persist for a long time. Thus, that is why they are called "long memory" processes.

Here is a great book that gives an introductory explanation of how these processes work with minimum mathematics needed:

You can see the book here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.