2
$\begingroup$

This is the follow up to the question Taylor series of a convolution.

Continuing the derivation given at Probability Theory: The Logic Of Science By E. T. Jaynes, chapter 7 "The Central Gaussian, Or Normal, Distribution", p.706 The Landon derivation. The author continues:

At the same time, the expectation of $v^2$ is increased to $\sigma^2 + \langle \epsilon^2 \rangle$, so Landon's invariance property requires that $f(v)$ should be equal also to

$(1)\quad f(v) = p(v|\sigma) + \langle\epsilon^2 \rangle {\partial p(v|\sigma)\over \partial \sigma^2}$

Where:

$\langle\epsilon^2 \rangle = \int \epsilon^2 q(\epsilon)d\epsilon $

$\langle\epsilon^2 \rangle$ is the expected value of $\epsilon^2$

From the text before these equations:

Landon reasoned that if this frequency distribution of noise voltage is so universal, then it must be better determined theoretically than empirically. To account for this universality but for magnitude, he visualized not a single distribution for the voltage at any given time, but a hierarchy of distributions $p(v|\sigma)$ characterized by a single scale parameter $\sigma^2$, which we shall take to be the expected square of the noise voltage. The stability seems to imply that if the noise level $\sigma^2$ is increased by adding a small increment of voltage, the probability distribution still has the same functional form, but only moved up the hierarchy to the new value of $\sigma$. He discovered that for only one functional form of $p(v|\sigma)$ will this be true.

Suppose the noise voltage $v$ is assigned the probability distribution $p(v|\sigma)$. Then it is incremented by a small extra contribution $\epsilon$, becoming $v' = v + \epsilon$ where $\epsilon$ is small compared to $\sigma$, and has a probability distribution $q(\epsilon)d\epsilon$, independent of $p(v|\sigma)$. Given a specic $\epsilon$, the probability for the new noise voltage to have the value $v'$ would be just the previous probability that $v$ should have the value $(v' - \epsilon)$.

Questions:

  1. What exactly is the Landon invariance property?
  2. How is that used to derive equation (1) above?
$\endgroup$

1 Answer 1

3
$\begingroup$

I'm working through this book as well and I'm no expert on these things so my explanation may be a bit crude, but I'll offer what insight I have.

For question 1, on page 205 in paragraph 3 Jaynes writes:

The stability seems to imply that, if the noise level $\sigma^2$ is increased by adding a small increment of voltage, the probability distribution still has the same functional form, but is only moved up the hierarchy to the new value of $\sigma$.

From this I infer that Landon's invariance property is the invariance of the functional form of the distribution.

To answer question 2, the derivation of $(1)$ does not require this property. We just perform a basic calculus argument. Let $\left<\epsilon^2\right>$ be that small increment of voltage and denote $f(v)$ to be the new distribution acquired from this small change. Then we note $$\lim_{\left<\epsilon^2\right>\rightarrow0}f(v)=p(v|\sigma^2).$$ From the difference quotient definition of a derivative, we have $$\lim_{\left<\epsilon^2\right>\rightarrow0} \frac{f(v)-p(v|\sigma^2)}{\left<\epsilon^2\right>}= \frac{\partial p(v|\sigma^2)}{\partial\sigma^2}.$$ Then, for small $\left<\epsilon^2\right>$ we have the approximation $$f(v)\approx p(v|\sigma^2)+\left<\epsilon^2\right>\frac{\partial p(v|\sigma^2)}{\partial\sigma^2}.$$

Landon's invariance property sets $f(v)$ equal to the right hand side of $(7.20)$.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .