# There's no way one can define the covariance matrix of just a probability measure without having a random variable, right? A paper did just that.

I'm reading a paper (that I'm unable to share for privacy reasons, sorry!) and for the first time, I came across the term "covariance matrix of a probability measure $$\mu$$ on a Euclidean space $$\mathbb{R}^m$$" without any further explanation. I'm having trouble understanding what the author meant, and since the author doesn't work in this area anymore, I can't get a response from her/him.

Here's my way of interpreting it, but I'm seemingly running into an issue:

Attempt to interpret: We say that the random vector defined on a probability space, i.e. $$X:(\Omega, \mathcal{F},P)\to\mathbb{R}^m$$ "follows"($$\sim$$) the probability distribution induced by $$\mu$$ if $$P(X\in E):=\mu(E) \forall \text{ measurable } E\subset \mathbb{R}^m.$$ Once $$X$$ is specified, we can attempt to define its mean vector and covariance matrix in the usual way: namely $$\mu:=E[X]:=\int_{\Omega}XdP, cov[X]:=E[(X-\mu)][(X-\mu)^{T}].$$ But the problem is: how can I write $$E[X], cov[X]$$ in terms of $$\mu$$ only, without having to depend on $$X?$$ If we know that the Radom Nikodym derivative $$f:=\frac{dP}{d\mu}$$ of $$P$$ w.r.t. $$\mu$$ exists then we can write $$\mu:=E[X]:=\int_{\Omega}XdP= \int_{\mathbb{R}^m}xf(x)d\mu(x),$$ where this $$f$$ is the density of $$X$$ w.r.t. $$\mu$$ that then exists. So we can write:

$$\text{ **Mean of a probability measure** }"E[\mu]":=\int_{\mathbb{R}^m}xf(x)d\mu(x)$$

$$\text{ **Covariance matrix of a probability measure** }"cov[\mu]":=\int_{\mathbb{R}^m}(x-\mu)(x-\mu)^{T}f(x)d\mu(x).$$

But then, we face two problems:

Issue 1) How to define the mean and covariance of the measure $$\mu$$ when the Radom Nikodym derivative $$f:=\frac{dP}{d\mu}$$ of $$P$$ w.r.t. $$\mu$$ does not exist?

Issue 2) The $$E[\mu], cov[\mu]$$ still depends on the probability measure $$P$$ on $$\Omega,$$ because $$f:=dP/d\mu$$ depends on $$P.$$ So it doesn't seem like a valid definition.

This is perhaps relevant to note that, earlier, I used to interpret:

"mean of a probability measure $$\mu$$ on a probability space $$(\Omega, \mathcal{F},\mu)$$ as a function of one random variable:

$$X\mapsto \int_{\Omega}Xd\mu$$

and

"covariance of a probability measure (Relevant Wiki article) $$\mu$$" in the same setting as a function acting on two random variables:

$$(X,Y)\mapsto \int_{\Omega}(X-\mu)][(X-\mu)^{T}d\mu.$$

But in these cases, $$\mu$$ didn't need to be defined necessarily on $$\mathbb{R}^m,$$ or in general, on a metric or topological space.

I think there's no hope to define it, because even intuitively, "covariance" is supposed to measure fluctuation of a function, and with just a measure present, whose fluctuations are we really measuring? But correct me if I'm wrong!

• Not sure there's a lot of point guessing what some author intended when we don't have the reference in hand. I agree that there's no obvious meaning that might have been intended, but of course context might clear that up. Or it might not...some papers are just badly written.
– lulu
Commented Nov 22, 2023 at 12:55
• @lulu Yes I agree with you here, but unfortunately there's a clause that I signed that prevents me from posting anything that's mentioned in the paper itself. Commented Nov 22, 2023 at 12:57
• Probability measures are identified with random variables. To wit, $m = \int x\mu(dx)$ and $\Sigma = \int (x-m)(x-m)^T \mu(dx)$ Commented Nov 22, 2023 at 12:57
• And if you want to go about your approach, how do you think $X$ is constructed? What do you think $\Omega$ is? 2) As written, $dP/d\mu$ is meaningless. (What is a RN derivative?) 3)It's a basic result (often called change of variables) that $\int_{\Omega}X\,dP = \int_{\mathbb R^d}x\mu(dx)$ if $\mu$ is the pushforward Commented Nov 22, 2023 at 13:00
• Let $\mu(\mathbb R^d) = 1$. Take $\Omega = \mathbb R^d,\mathcal F = \mathcal B(\mathbb R^d),P = \mu$ and $X = \mathrm{Id}$ Commented Nov 22, 2023 at 13:11

Let $$\mu$$ be a measure on $$(\mathbb R^d, \mathcal B(\mathbb R^d))$$. Under an integrability assumption, the mean of $$\mu$$ can be defined as $$E[\mu] = \int_{\mathbb R^d} x d\mu(x)$$, where the integral is a Bochner integral.
Similarly, under an appropriate integrability assumption, the covariance matrix is the Bochner integral $$cov(\mu) = \int_{\mathbb R^d} (x-E[\mu])(x-E[\mu])^{\top} d\mu(x)$$.
Let $$(\Omega, \mathcal F, \mathbb P)$$ denote a probability space and $$X:(\Omega, \mathcal F)\to (\mathbb R^d, \mathcal B(\mathbb R^d))$$ be a random vector. The image measure of $$X$$ is defined as $$\mathbb P_X:B\mapsto \mathbb P(X\in B)$$. $$X$$ is said to have distribution $$\mu$$ if $$\mathbb P_X = \mu$$.
• thank you for your definitions, they do make sense! Quick question though: I see that to define the mean and covariance of a probability measure (in fact, it didn't have to be a probability measure it seems, since these integrals can be defined without requiring $\mu(R^d)=1.$), you did not need a probability space or a random vector whose domain is that prob. space and codomain is the Euclidean space. I udnerstand this. But when you do define $X$ so that $\mathbb{P}(X\in B):=\mu(B),$ then we can also attempt to define $E[X]:=\int_{\Omega}XdP.$ Is $E[X]=E[\mu]?$ . Commented Nov 22, 2023 at 13:21
• (continued) I think $E[X)=E[\mu]$ follows from the change of variable formula for measures that Andrew was mentioning in his comments? Commented Nov 22, 2023 at 13:23
• @LearningMath Yes, you can write the Bochner integral $E[X] = \int_{\Omega} X(w) d\mathbb P(w)$. By the law of the unconscious statistician (which also holds for Bochner integrals), you get indeed that $E[X] = E[\mu]$. Commented Nov 22, 2023 at 13:31