# Apparent contradiction for probability density functions?

Consider a probability density function $\it{pdf}$, $f\left(x\right)$, which can be expanded as:

$$f\left(x\right) = \sum_{k=1}^{\infty} \alpha_k \delta\left(x-x_k\right)$$

It is easy to verify, by integrating both sides that, $\int_{-\infty}^{\infty}f\left(x\right) = 1$, implies $\sum_{k=1}^{\infty}\alpha_k=1$. Also the orthogonality of dirac delta functions centered on distinct points $x_k$ and $x_m$ leads to $\alpha_k=f\left(x_k\right)$. Here's the problem, the only bound on a $\it{pdf}$ is non-negativity, so since $f\left(x\right) \in \left[0,\infty\right)$, we should have $\alpha_k \in \left[0,\infty\right)$. If $\alpha_k$ has no upper bound, how can they sum to $1$?

-

Your implication $\sum_{k=1}^{\infty}\alpha_k=1$ is correct. The problem is your assumption $\alpha_k=f(x_k)$. This is not the case, since the delta function is defined by its integral; it is a distribution and does not have a function value in the traditional sense. So evaluating $\delta(x-x_k)$ at $x=x_k$ is meaningless.

EDIT: This is a reaction to your example in the comment below. $$f(x_m)=\int_{-\infty}^{\infty}f(x)\delta(x-x_m)\;dx = \int_{-\infty}^{\infty}\sum_{k=1}^{\infty}\alpha_k\delta(x-x_k)\delta(x-x_m)\;dx$$ Assuming that we can interchange integration and summation, we get $$f(x_m) = \sum_{k=1}^{\infty}\alpha_k\int_{-\infty}^{\infty}\delta(x-x_k)\delta(x-x_m)\;dx$$ Now we evaluate the integral in a way that is common in physics and engineering; I believe mathematicians usually frown upon such manipulations (please correct me if I'm mistaken): $$f(x_m) = \sum_{k=1}^{\infty}\alpha_k\delta(x_m-x_k)\tag{1}$$ which is not the same expression that you derived in your comment. Note that (1) is exactly what you get when you evaluate $f(x)$ at $x=x_m$ in your original expression for $f(x)$. So $f(x_m)\neq \alpha_m$, because here, $\delta(x)$ is not the Kronecker delta, but the delta function.

-
Not as a Riemann integral. But when interpreted in lebesgue sense, it is ok. In any case, all integrals in probability theory are Lebesgue Integrals. – Gautam Shenoy Apr 23 '13 at 20:18
Yes, but that's not what Matt meant. What he's saying is that the "value" of a Dirac delta is defined only in terms of its impact on the integral. Colloquially, its value is infinity at its centerpoint. Either way, $\alpha_k=f(x_k)$ is not correct. – Michael Grant Apr 24 '13 at 0:42
I think it's because it's a dirac delta function. Instead if it were to be interpreted as $1_{x-x_k}$, he'd be fine. – Gautam Shenoy Apr 24 '13 at 12:13
@Matt: I'm not following what is being said regarding $\alpha_k \ne f\left(x_k\right)$. Take my original definition of $f\left(x\right)$, and multiply both sides by $\delta\left(x_m\right)$ and integrate w.r.t. $x$: – okj Apr 24 '13 at 13:31
$$\begin{array}{rcl} \displaystyle\int_{-\infty}^{\infty}f\left(x\right)\delta\left(x-x_m\right)\text‌​{d}x &= &\displaystyle\sum_{k=1}^{\infty}\int_{-\infty}^{\infty}\delta\left(x-x_k\right)‌​\delta\left(x-x_m\right)\text{d}x \\ f\left(x_m\right) &= &\displaystyle\sum_{k=1}^{\infty}\alpha_k\delta_{m,k} \\ f\left(x_m\right) &= &\alpha_m \end{array}$$ – okj Apr 24 '13 at 13:40

Matt has correctly pointed out one problem with your question, namely that it makes no sense to evaluate a distribution (as opposed to a function) at a point. There is, however, an additional problem with the question. You wrote "the only bound on a pdf is nonnegativity". That's the only bound that refers to values of a pdf at just one point at a time (when the pdf is a function, not merely a distribution, so that it has values at points), but the requirement that the integral be $1$ amounts to an additional, global bound. You can't just have arbitrarily large values everywhere and expect the integral to be 1. A pdf can have an arbitrarily large value at one point (or at a few points) but it will have to have small values elsewhere to make the integral $1$.

-

But it does!! Since $$\int_{-\infty}^{\infty} f(x)dx =1 \Rightarrow \sum_{k=1}^{\infty} \alpha_k = 1$$

Now we have $\alpha_k = f(x_k) \geq 0$, and since $\sum_{k=1}^{\infty} \alpha_k$ is a convergent series, we also have $\alpha_k \to 0$. Moreover

$$\alpha_n \leq \sum_{k=1}^{\infty} \alpha_k = 1$$

Thus $0 \leq \alpha_n \leq 1$ for all n.

By chance I have misunderstood your question this might help:

$$\alpha_k \delta{\{x-x_k\}} \leq \sum_{k=1}^{\infty} \alpha_k\delta{\{x-x_k\}} = f(x)$$ Integrate both sides to get (Noting all terms are nonnegative) that $$\alpha_k \leq 1$$

-
This follows the question in erroneously assuming that one can evaluate a distribution at an arbitrary point, and specifically $\delta(0)=1$. Also, I cannot make sense of your comment on Matt's answer, about Riemann versus Lebesgue integrals, since the only integral mentioned by Matt is the one implicit in the very notion of distribution. – Andreas Blass Apr 24 '13 at 0:38
Which is why I prefer working with cdf's as far as possible. As for the integral comment I made, I was thinking in terms of simple functions. Even though OP had clearly mentioned Dirac Delta functions, my proof is correct if we take the dirac delta to be an indicator function. Since it isn't it's wrong. Ill keep this answer however so that others can debate on it. – Gautam Shenoy Apr 24 '13 at 12:16