# Probability distributions with memory

While reading the book "Algorithms to live by" by Brian Christian and I came across the concept of probability distributions with memory. In the text they mention the power law distribution as an example.

Doing some online search I did not find much information about the topic:

• what is the definition of a probability distribution with memory?
• Why a power law distribution is a prob. distribution with memory?
• or, what are the basic properties of these distributions...

I only found some non very generic papers on the topic.

I wonder if anyone has knowledge about this or if someone could recommend some useful and accessible literature to a non-expert.

Usually, when someone talks of "long memory" typically they are referring to a a time series process with an unbounded spectrum as frequency goes to zero and is called a “long memory process.” A class of processes with this property are the “fractionally integrated” series generated by

$$(1-B)^dX_t = Y_t,$$

with $B$ the lag operator and $Y_t$ is a stationary series and $X_t$ as a fractionally integrated series.

For these types of time series,

$$E[X_t] \sim m t^d,$$

and the autocorrelation function

$$\rho(k) \sim c k ^{2d-1}$$

possesses "long memory" because the autocorrelations DO NOT decay exponentially (like typical processes do when their variance < $\infty$) - they decay via a power-law and persist for a long time. Thus, that is why they are called "long memory" processes.

Here is a great book that gives an introductory explanation of how these processes work with minimum mathematics needed:

You can see the book here