5
$\begingroup$

My apologies if this is a naive question but does anyone happen to know if the equation below (and/or the distribution it describes) is commonly know by a specific name?

$f(i) = ( a / i^k ) b^i$

(where $a$, $b$ and $k$ are constants; and, FWIW, for my purposes $i$ should be a positive integer denoting rank according to frequency)

This equation appears in Simon (1955), but he doesn't give it a specific name and always refers to it by its example number in his paper.

This is not my field so I'm not familiar with the relevant literature so I don't know how often this particular distribution crops up and if it is common enough to warrant a specific name (pointers to places where it does appear would also be appreciated though!).

The same formula is also mentioned in Tambovtsev & Martindale (2007), who call it a "Yule distribution" but, as far as I can glean, this seems wrong since it appears "Yule(-Simon)" refers to a different distribution; T&M seem to have mislabelled it based on a careless reading of Simon, who does indeed mention the above equation in the proximity of a discussion of Yule (1924).

Instead, Simon cites Champernowne (1953) for this equation; however, calling it a "Champernowne distribution" doesn't seem appropriate either since, isn't that already the name for a different distribution?

Help, comments and corrections would be gratefully appreciated!

References

Champernowne, D. G. 1953. A Model of Income Distribution. The Economic Journal 63(250): 318–51

Simon, H. A. 1955. On a class of skew distribution functions. Biometrika 42(3–4). 425–40.

Tambovtsev, Y. A. & C. Martindale. 2007. Phoneme frequencies follow a Yule distribution. SKASE Journal of Theoretical Linguistics 4(2). 1–11.

Yule, G. U. 1924. A mathematical theory of evolution, based on the conclusions of Dr J. C. Willis, F.R.S. Philosophical Transactions B 213(21).

$\endgroup$
2
+100
$\begingroup$

As far as I can tell, it doesn't quite have a name, but perhaps the following makes sense? The logarithmic distribution has pmf \begin{align*} p(i) = -\frac{1}{\log(1-p)}\frac{p^i}{i} \end{align*} So this would be a special case when $k = 1$ in your parametrization, and $a$ is then determined to ensure the pmf sums to 1. The reason for this name is because the pmf is derived from the series expansion of the logarithmic function. In your formulation, the pmf would indeed equal \begin{align*} p(i) = \text{Li}_{k}(b) \frac{b^i}{i^k} \end{align*} Hence, if we would to continue the tradition of naming distributions based off series expansions from which they are derived, this could be called the "Polylogarithmic distribution". No results show up when I google this, though.

$\endgroup$
  • $\begingroup$ Thank you for this! I've managed to find uses of "polylogarithmic distribution" in the wild so it seems there is precedent in the literature to use this name. For example, if you look at this (appliedprobability.org/data/files/TMS%20articles/20_2_4.pdf) paper, equation (4) looks equivalent to the equation in my question (i.e. (1.1) in Simon 1955) and he uses the name "polylogarithmic distribution" to refer to it in the rest of the paper. $\endgroup$ – Miztli Apr 3 at 14:15
  • $\begingroup$ Ah, when I include quotes, I do find some results now. Thank you, and happy to help! $\endgroup$ – Tom Chen Apr 4 at 11:56
2
$\begingroup$

Assuming that $a$ is a normalization constant, the formula $$ \label{eq:original}\tag{1} f(i) = \frac{ab^i}{i^k} $$ is mentioned by Good (1953, eqn 54) as a generalization of the Zipf distribution $$ f(i) = \frac{a}{i^k}. $$ (Good stipulates $0 < b < 1$ and calls $b^i$ a "convergence factor".) There is precedent in the literature for calling ($\ref{eq:original}$) the "Good distribution": see Zörnig & Altmann (1995) and Eeg-Olofsson (2008).

A more general form, $$ f(i) = \frac{ab^i}{(\nu + i)^k}, $$ has been dubbed the "Lerch distribution" (Zörnig & Altmann 1995; Klar, Parthasarathy & Henze 2010, p. 130), as the normalization constant is related to the Lerch transcendent.

I can't think of an obvious/objective criterion for choosing between "Good distribution" and "polylogarithmic distribution" (Tom Chen's answer; Kemp 1995) – while both names appear in the literature, the references are so few it's not clear that one name should be favoured over the other. Perhaps one could call it the Good–Kemp distribution?

References

Eeg-Olofsson, M. (2008) Why is the Good distribution so good? Towards an explanation of word length regularity. Lund University Department of Linguistics and Phonetics Working Papers, 53, 15–21. https://journals.lub.lu.se/LWPL/article/view/2270/1845

Good, I. J. (1953) The population frequencies of species and the estimation of population parameters. Biometrika, 40, 237–264. https://doi.org/10.1093/biomet/40.3-4.237

Kemp, A. W. (1995) Splitters, lumpers and species per genus. Mathematical Scientist, 20, 107–118. http://www.appliedprobability.org/data/files/TMS%20articles/20_2_4.pdf

Klar, B., Parthasarathy, P. R. & Henze, N. (2010) Zipf and Lerch limit of birth and death processes. Probability in the Engineering and Informational Sciences, 24, 129–144. https://doi.org/10.1017/S0269964809990179

Zörnig, P. & Altmann, G. (1995) Unified representation of Zipf distributions. Computational Statistics & Data Analysis, 19, 461–473. https://doi.org/10.1016/0167-9473(94)00009-8

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.