Difference between power law distribution and exponential decay

Question

This is probably a silly one, I've read in Wikipedia about power law and exponential decay. I really don't see any difference between them. For example, if I have a histogram or a plot that looks like the one in the Power law article, which is the same as the one for $e^{-x}$, how should I refer to it?

Answer 1

$$ \begin{array}{rl} \text{power law:} & y = x^{(\text{constant})}\\ \text{exponential:} & y = (\text{constant})^x \end{array} $$

That's the difference.

As for "looking the same", they're pretty different: Both are positive and go asymptotically to $0$, but with, for example $y=(1/2)^x$, the value of $y$ actually cuts in half every time $x$ increases by $1$, whereas, with $y = x^{-2}$, notice what happens as $x$ increases from $1\text{ million}$ to $1\text{ million}+1$. The amount by which $y$ gets multiplied is barely less than $1$, and if you put "billion" in place of "million", then it's even closer to $1$. With the exponential function, it always gets multiplied by $1/2$ no matter how big $x$ gets.

Also, notice that with the exponential probability distribution, you have the property of memorylessness.

Answer 2

How is a power law different from an exponential? (I'm putting this answer here mainly for my own future reference. Hopefully someone else may find it useful.)

Power Law function
(notice the exponent, $k,$ is a constant) $$ y = x^k $$

Exponential function
(notice the exponent is a variable) $$ y = a^x $$

Technical definition of Power Law:

A power law is any polynomial relationship that exhibits the property of scale invariance.

Scale invariance (from Wikipedia)

One attribute of power laws is their scale invariance. Given a relation $f(x) = ax^k,$ scaling the argument $x$ by a constant factor $c$ causes only a proportionate scaling of the function itself. That is,

$$ f(c x) = a(c x)^k = c^k f(x) \propto f(x) $$

That is, scaling by a constant $c$ simply multiplies the original power-law relation by the constant $c^k.$

Thus, it follows that all power laws with a particular scaling exponent are equivalent up to constant factors, since each is simply a scaled version of the others. This behavior is what produces the linear relationship when logarithms are taken of both $f(x)$ and $x,$ and the straight-line on the log-log plot is often called the signature of a power law.

Answer 3

If there is anybody landing on this from Nassim Nicholas Taleb's The Black Swan the issue at stake is how doubling a random variable affects the probability in power law distributions as opposed to a normal or Gaussian distribution.

In the case of continuous random variables, the pdf illustrates the difference:

A power law distribution has a pdf of the form

$$f_X(x) =C x^{-\alpha},$$

where $C$ is a constant, and $\alpha$ is the law's exponent.

The effect of doubling $x$ will remain constant across the domain: For example, the ratio in the pdf between people who make $\$50,000$ per year to those that make $\$25,000$ will be the same as the ratio between people that make $\$10,000,000$ to those that make $\$5,000,000:$

$$\frac{(2x)^{-\alpha}}{x^{-\alpha}}=2^{-\alpha}$$

an attribute called scale invariance.

This is in contrast to the rapid decay in the normal distribution, which in the standardized form has the following pdf:

$$f_X(x)=\frac{1}{\sqrt{2\pi}}\exp\left( -\frac{x^2}{2}\right)$$

Doubling the value of $x,$ amounts to raising to the third power the exponential (un-normalized) part of the pdf:

$$\begin{align} \frac{f_X(2x)}{f_X(x)}&=\exp\left(-\frac{(2x)^2}{2} +\frac{{}x^2}{2}\right)\\[2ex] &=\exp\left( -\frac{3x^2}{2}\right)\\[2ex] &=\left(\exp\left(-\frac{x^2}{2} \right)\right)^{3}\\[2ex] &=\frac{1}{\left(\mathrm e^{x^2/2}\right)^3} \end{align}$$

which can be visually plotted as:

Therefore, at asymptotic values the relative probability between an extreme event and that of its value doubled will rapidly tend to zero, whereas in a power law it will remain stubbornly constant.

Answer 4

If you have a limited data set, one way to tell the difference is to put the data into spreadsheet software capable of exponential and power regressions and see which gives the better correlation coefficient. Presumably the coefficient is calculated by comparing the least squares errors of the semi-log and log-log plots. A bit more on that...

Let's call an exponential law one like $y = Ca^x$ and a power function one like $y = Cx^p$. If we take the logarithm of both sides of an exponential function, we get $$ \log y = \log C + x \log a. $$ That is, the collection of ordered pairs $(x, \log y)$ (the semi-log plot) should be roughly linear for exponential data.

On the other hand, for a power function we get $$ \log y = \log C + p \log x, $$ so the collection of ordered pairs $(\log x, \log y)$ (the log-log plot) should be roughly linear for power law data.

Determining which of these two plots is more line-like can tell whether exponential or power laws best model the original data.

Answer 5

very different. A power law just says that some variable is a power of the other. For example, in physics

$$y=3x^2$$

is a power law between $y$ and $x$ where the power is $2$ (the coefficient doesn't matter).

$$y=x^2+x$$

is not. It must be one term of the form $cx^n$.

Exponential decay, on the other hand, is a similar idea, but formed around $Ce^{-kt}$ instead, for some constants $c$ and $k$.

The image in the wikipedia page on the power law is probably something like $\frac 1 x$, not an exponential decay curve.