Prove an interesting property of pdf moments?

I am examining properties of mass moments of probability densities:

$$m_{i}\equiv\int_{-\infty}^{\infty}x^{i}f\left(x\right)dx$$

Define a $\,n\times n\,$ covariance of the first $\,n\,$ moments:

$$M_{ij}\equiv\int_{-\infty}^{\infty}f\left(x\right)\left(x^{i}-m_{i}\right)\left(x^{j}-m_{j}\right)dx=m_{i+j}-m_{i}m_{j}$$ where $1\leq i\leq n$ and $1\leq j\leq n$

I suspect but have failed to prove that:

$$\left|M_{n\times n}\right|=0\iff\left\{ \begin{array}{c} f\left(x\right)=\frac{1}{r}\sum_{j=1}^{r}\delta\left(x-x_{j}\right)\\ r\leq n\end{array}\right\}$$

In other words if $M$ is $n\times n$ and its Determinant is $0$, then $f(x)$ is a set of $n$ or fewer spikes.
If $f(x)$ has more than $n$ spikes or has any continuous portion then $|M|>0$ for all $n$.

It's easy enough to demonstrate for chosen values of $r$ that $|M|=0$ ; the Mathematica code below does that. But I need a proof that if $\,|M|=0$ , then $\,f(x)\,$ consists of $\,r\,$ spikes, where $\,r\le n$ .

(*Demonstrate that for any n, Det[M]==0 if r==n, Det[M]>0 if r>n *)

n = 4;
M = Table[m[i + j] - m[i]*m[j], {i, 1, n}, {j, 1, n}];
Print[n, "-particle Det = ", Det[M] /. m[q_] -> (Sum[x[i]^q, {i, 1, n}]/n) // Simplify];
Print[n + 1, "-particle Det = ", Det[M] /. m[q_] -> (Sum[x[i]^q, {i, 1, n + 1}]/(n + 1)) // Simplify];
• Maybe you can use the fact that $m[j]=\sum_{i=1}^n \frac{x_i^j}{n}$. – Bernhard Sep 11 '15 at 6:43
• @Bernhard Yes, that's exactly what I do in the demo code. It demonstrates that Det[M] is zero for whatever value of n you use, but I need a general proof that this is true for all n. – Jerry Guern Sep 11 '15 at 6:57
• @Bernhard I just posted a bounty on this Quesiton – Jerry Guern Sep 13 '15 at 12:22
• Do the zeroth moments count as well? I mean, is: $0\le i$ and $0\le j$ ? – Han de Bruijn Sep 13 '15 at 14:58
• @HandeBruijn Not sure what you meant, but f(x) is a pdf, so the 0th moment is just 1. – Jerry Guern Sep 13 '15 at 15:03

It's false as stated: the determinant should be $0$ for any $f$ that is supported on a set of cardinality $\le n$, not just a uniform distribution. In fact, let $X$ be a random variable whose distribution is supported on a set $\{r_1, \ldots, r_n\}$ of cardinality $n$. Let $P(z) = \prod_{k=1}^n (z - r_k) = \sum_{j=0}^n a_j z^j$. If ${\bf a} = (a_1, \ldots, a_n)^T$, then $P(X) = 0$ so $(M {\bf a})_j = \text{Cov}(X^j, P(X)) = 0$ (the lack of an $a_0$ term does not matter because $\text{Cov}(X^j, 1) = 0$) and therefore $M$ is singular.

Conversely, if $M$ is singular, take ${\bf b} \ne 0$ in its null space, and define $P(z) = \sum_{k=1}^n b_j z^j$. We have $\text{Var}(P(X)) = \text{Cov}(P(X),P(X)) = {\bf b}^T M {\bf b} = 0$, which implies $P(X) = 0$ a.s., and that means $X$ is almost surely one of the roots of $P$ (of which there are at most $n$).

• A very dense answer! Thank you. I gather that you're saying M will be singular even if the spikes in the pdf have unequal weight. But I'm struggling with this step in your explanation: $(M {\bf a})_j = \text{Cov}(X^j, P(X)) = 0$. Could you give that a bit of explanation? Also, I'm not certain if your answer proves $M_{n\times n}$ is non-singular if f(x) has more than n spikes. – Jerry Guern Sep 13 '15 at 16:07
• Covariance is bilinear, so $\text{Cov}(X^j, P(X)) = \sum_{k=0}^n a_k \text{Cov}(X^j, X^k) = \sum_{k=1}^n M_{jk} a_k = (M {\bf a})_j$. – Robert Israel Sep 13 '15 at 16:17
• Thank you! It took me some time to understand this, hence the delay in awarding the bounty. – Jerry Guern Sep 14 '15 at 4:21
• I posted a more baby-steps version of this answer below. – Jerry Guern Sep 14 '15 at 4:30
• One more small question. What does "almost surely" mean in math? I can't even find a good online explanation, and I've looked. – Jerry Guern Sep 17 '15 at 2:05

I posted the Question and awarded the bounty to @RobertIsrael for his great Answer above. But I had to think pretty hard about his answer and "translate" into simpler language for myself, so I thought I'd post my simpler version.

First of all, my assertion wasn't quite correct because the spikes in the pdf $f(x)$ can be differently weighted. So the corrected theorem is:

$$\left|M_{n\times n}\right|=0\iff\left\{ \begin{array}{c} f\left(x\right)=\sum_{j=1}^{r}w_{j}\delta\left(x-x_{j}\right)\\ r\leq n\end{array}\right\}$$

Since any weight can be zero, the case where there are fewer than n spikes is covered. The RHS of the above eqn can be restated as:

$$f\left(x\notin\left\{ x_{1},...,x_{n}\right\} \right)=0$$

Now consider this n-th order polynomial:

$$q\left(x\right)\equiv\prod_{j=1}^{n}\left(x-x_{j}\right)=\sum_{k=0}^{n}a_{k}x^{k}$$

It is trivial but not necessary to calculate the $a_k$'s. For any x, either $f(x)$ or $q(x)$ will equal 0. Therefore, the following integrand and integral both equal zero:

$$0=\int f\left(x\right)\left(x^{j}-m_{j}\right)q\left(x\right)dx=\int f\left(x\right)\left(x^{j}-m_{j}\right)\sum_{k=0}^{n}a_{n}x^{k}dx=\sum_{k=0}^{n}a_{n}\left(m_{j+k}-m_{j}m_{k}\right)=\left(M_{n\times n}a\right)_{j}$$

This proves that $a$ is a 0-eigenvector of M, which can only occur if M is singular, QED: $$f\left(x\notin\left\{ x_{1},...,x_{n}\right\} \right)=0\Rightarrow|M_{n\times n}|=0$$

Proof of the converse: If $M$ is singular, then there exists some $a\neq0$ such that $Ma=0$, from which we can construct a polynomial $q\left(x\right)\equiv\sum_{k=0}^{n}a_{k}x^{k}$, the covariance of which is:

$$Cov\left(q\left(x\right)\right)=\int f\left(x\right)\left(q\left(x\right)-\left\langle q\left(x\right)\right\rangle \right)^{2}=\int f\left(x\right)\left(\sum_{k=0}^{n}a_{k}\left(x^{k}-m_{k}\right)\right)^{2}=a^{T}Ma=0$$

$Cov(q(x))=0$ implies that $f(x)=0$ whenever $q(x)\neq0$, i.e., $f(x)$ can only be non-zero at the n roots of $q(x)$, QED:

$$|M_{n\times n}|=0\Rightarrow f\left(x\notin\left\{ x_{1},...,x_{n}\right\} \right)=0$$

...which completes the proof.

• Hey, thanks for the up-vote. I'm glad to know someone found this extra explanation helpful. – Jerry Guern Sep 16 '15 at 6:31