1
$\begingroup$

I'm reading a paper (that I'm unable to share for privacy reasons, sorry!) and for the first time, I came across the term "covariance matrix of a probability measure $\mu$ on a Euclidean space $\mathbb{R}^m$" without any further explanation. I'm having trouble understanding what the author meant, and since the author doesn't work in this area anymore, I can't get a response from her/him.

Here's my way of interpreting it, but I'm seemingly running into an issue:

Attempt to interpret: We say that the random vector defined on a probability space, i.e. $X:(\Omega, \mathcal{F},P)\to\mathbb{R}^m$ "follows"($\sim$) the probability distribution induced by $\mu$ if $P(X\in E):=\mu(E) \forall \text{ measurable } E\subset \mathbb{R}^m.$ Once $X$ is specified, we can attempt to define its mean vector and covariance matrix in the usual way: namely $\mu:=E[X]:=\int_{\Omega}XdP, cov[X]:=E[(X-\mu)][(X-\mu)^{T}].$ But the problem is: how can I write $E[X], cov[X]$ in terms of $\mu$ only, without having to depend on $X?$ If we know that the Radom Nikodym derivative $f:=\frac{dP}{d\mu}$ of $P$ w.r.t. $\mu$ exists then we can write $\mu:=E[X]:=\int_{\Omega}XdP= \int_{\mathbb{R}^m}xf(x)d\mu(x),$ where this $f$ is the density of $X$ w.r.t. $\mu$ that then exists. So we can write:

$$ \text{ **Mean of a probability measure** }"E[\mu]":=\int_{\mathbb{R}^m}xf(x)d\mu(x)$$

$$ \text{ **Covariance matrix of a probability measure** }"cov[\mu]":=\int_{\mathbb{R}^m}(x-\mu)(x-\mu)^{T}f(x)d\mu(x).$$

But then, we face two problems:

Issue 1) How to define the mean and covariance of the measure $\mu$ when the Radom Nikodym derivative $f:=\frac{dP}{d\mu}$ of $P$ w.r.t. $\mu$ does not exist?

Issue 2) The $E[\mu], cov[\mu]$ still depends on the probability measure $P$ on $\Omega,$ because $f:=dP/d\mu$ depends on $P.$ So it doesn't seem like a valid definition.

This is perhaps relevant to note that, earlier, I used to interpret:

"mean of a probability measure $\mu$ on a probability space $(\Omega, \mathcal{F},\mu)$ as a function of one random variable:

$X\mapsto \int_{\Omega}Xd\mu$

and

"covariance of a probability measure (Relevant Wiki article) $\mu$" in the same setting as a function acting on two random variables:

$(X,Y)\mapsto \int_{\Omega}(X-\mu)][(X-\mu)^{T}d\mu.$

But in these cases, $\mu$ didn't need to be defined necessarily on $\mathbb{R}^m,$ or in general, on a metric or topological space.

I think there's no hope to define it, because even intuitively, "covariance" is supposed to measure fluctuation of a function, and with just a measure present, whose fluctuations are we really measuring? But correct me if I'm wrong!

$\endgroup$
8
  • 2
    $\begingroup$ Not sure there's a lot of point guessing what some author intended when we don't have the reference in hand. I agree that there's no obvious meaning that might have been intended, but of course context might clear that up. Or it might not...some papers are just badly written. $\endgroup$
    – lulu
    Commented Nov 22, 2023 at 12:55
  • $\begingroup$ @lulu Yes I agree with you here, but unfortunately there's a clause that I signed that prevents me from posting anything that's mentioned in the paper itself. $\endgroup$ Commented Nov 22, 2023 at 12:57
  • $\begingroup$ Probability measures are identified with random variables. To wit, $m = \int x\mu(dx)$ and $\Sigma = \int (x-m)(x-m)^T \mu(dx)$ $\endgroup$
    – Andrew
    Commented Nov 22, 2023 at 12:57
  • 2
    $\begingroup$ And if you want to go about your approach, how do you think $X$ is constructed? What do you think $\Omega$ is? 2) As written, $dP/d\mu$ is meaningless. (What is a RN derivative?) 3)It's a basic result (often called change of variables) that $\int_{\Omega}X\,dP = \int_{\mathbb R^d}x\mu(dx)$ if $\mu$ is the pushforward $\endgroup$
    – Andrew
    Commented Nov 22, 2023 at 13:00
  • 1
    $\begingroup$ Let $\mu(\mathbb R^d) = 1$. Take $\Omega = \mathbb R^d,\mathcal F = \mathcal B(\mathbb R^d),P = \mu$ and $X = \mathrm{Id}$ $\endgroup$
    – Andrew
    Commented Nov 22, 2023 at 13:11

1 Answer 1

2
$\begingroup$

Let $\mu$ be a measure on $(\mathbb R^d, \mathcal B(\mathbb R^d))$. Under an integrability assumption, the mean of $\mu$ can be defined as $E[\mu] = \int_{\mathbb R^d} x d\mu(x)$, where the integral is a Bochner integral.

Similarly, under an appropriate integrability assumption, the covariance matrix is the Bochner integral $cov(\mu) = \int_{\mathbb R^d} (x-E[\mu])(x-E[\mu])^{\top} d\mu(x)$.

Note that these quantities make sense without existence of a density (no need for Radon-Nikodym) and without reference to any random vector.

Let $(\Omega, \mathcal F, \mathbb P)$ denote a probability space and $X:(\Omega, \mathcal F)\to (\mathbb R^d, \mathcal B(\mathbb R^d))$ be a random vector. The image measure of $X$ is defined as $\mathbb P_X:B\mapsto \mathbb P(X\in B)$. $X$ is said to have distribution $\mu$ if $\mathbb P_X = \mu$.

$\endgroup$
3
  • $\begingroup$ thank you for your definitions, they do make sense! Quick question though: I see that to define the mean and covariance of a probability measure (in fact, it didn't have to be a probability measure it seems, since these integrals can be defined without requiring $\mu(R^d)=1.$), you did not need a probability space or a random vector whose domain is that prob. space and codomain is the Euclidean space. I udnerstand this. But when you do define $X$ so that $\mathbb{P}(X\in B):=\mu(B),$ then we can also attempt to define $E[X]:=\int_{\Omega}XdP.$ Is $E[X]=E[\mu]?$ . $\endgroup$ Commented Nov 22, 2023 at 13:21
  • $\begingroup$ (continued) I think $E[X)=E[\mu]$ follows from the change of variable formula for measures that Andrew was mentioning in his comments? $\endgroup$ Commented Nov 22, 2023 at 13:23
  • 1
    $\begingroup$ @LearningMath Yes, you can write the Bochner integral $E[X] = \int_{\Omega} X(w) d\mathbb P(w)$. By the law of the unconscious statistician (which also holds for Bochner integrals), you get indeed that $E[X] = E[\mu]$. $\endgroup$ Commented Nov 22, 2023 at 13:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .