When describing the Dirac delta function and Dirac distribution, my textbook says the following:
In some cases, we wish to specify that all the mass in a probability distribution clusters around a single point. This can be accomplished by defining a PDF using the Dirac delta function, $\delta(x)$:
$$p(x) = \delta(x - \mu)$$
The Dirac delta function is defined such that it is zero valued everywhere except $0$, yet integrates to $1$. The Dirac delta function is not an ordinary function that associates each value $x$ with a real-valued output; instead it is a different kind of mathematical object called a generalized function that is defined in terms of its properties when integrated. We can think of the Dirac delta function as being the limit point of a series of functions that put less and less mass on all points other than zero.
By defining $p(x)$ to be $\delta$ shifted by $- \mu$ we obtain an infinitely narrow and infinitely high peak of probability mass where $x = \mu$.
Goodfellow, Ian; Bengio, Yoshua; Courville, Aaron; Bach, Francis. Deep Learning (NONE) (Page 64). The MIT Press. Kindle Edition.
- I'm not sure that I understand what is meant by this part:
We can think of the Dirac delta function as being the limit point of a series of functions that put less and less mass on all points other than zero.
If someone could please demonstrate what this is (mathematically), then I'd greatly appreciate it.
- If we obtain an infinitely narrow and infinitely high peak, as shown in this Wikipedia article, then wouldn't the integral over that peak be equal to $0$, since the peak is infinitely narrow and therefore has no area underneath it? This is my understanding from my study of the Riemann integral.
Thanks for any clarification.