I am trying to get my head wrapped around this article in Wikipedia. The first definition given there is the covariance of a probability measure $\mathbf{P}$:

$$\mathrm{Cov}(x, y) = \int_{H} \langle x, z \rangle \langle y, z \rangle \, \mathrm{d} \mathbf{P} (z) \tag{$\ast$}$$

where $x, y \in H$ (a Hilbert space). I am used to the following definition of covariance:

$$\mathrm{Cov}(X, Y) = E((X - EX)(Y - EY))$$

where $X: \Omega \to \mathbb{R}$ and $Y: \Omega \to \mathbb{R}$ are two random variables defined on the corresponding probability space $(\Omega, \mathcal{F}, \mathbf{P})$, and

$$EX = \int_{\Omega} X(\omega) \, \mathrm{d} \mathbf{P} (\omega).$$

Question 1: Can anybody please translate my definition to the one in $(\ast)$? I would like to see it rewritten in the form of $(\ast)$, and I would really appreciate some discussion to get a better understanding of what is going on there.

At the end of the article, there is a definition of the covariance function of a random element $z$:

$$\mathrm{Cov}(x, y) = \int z(x) z(y) \, \mathrm{d} \mathbf{P} (z) = E(z(x) z(y)) \tag{$\ast\ast$}.$$

Question 2: First of all, why don't we subtract the expected values of $z$ at $x$ and $y$? Secondly, again, I do not really see a connection with my definition. Can anybody please give a clarification?

Thank you!

Regards, Ivan


1 Answer 1


The first definition is a special case of the second. Rather than a Hilbert space H, let's look at a Banach space $X$, and distinguish it from its dual space $X^*$. Every continuous linear functional $\varphi \in X^*$ is a random variable $\varphi : X \to \mathbb R$, so it makes sense to take expectations and covariances. We define the expectation of a functional by $\mathbb E[\varphi] = \int_X \varphi[x] \, \mathrm d \mathbf P(x)$, and the covariance of two functionals to be

$$\operatorname{cov}[\psi|\varphi] = \int_X \big( \psi[x] - \mathbb E[\psi]\big) \big( \varphi[x] - \mathbb E[\varphi]\big) \, \mathrm d \mathbf P(x).$$

Now, consider a probability measure $\mathbf P$ on a Hilbert space $X = H$. By the Riesz representation theorem, we know that the dual space $H^*$ is isomorphic to $H$, and all the functionals are of the form $\varphi_h[x] := \langle h, x \rangle$.

The mean can be represented by a single element $m \in H$, which is called the "Pettis integral" of $\mathbf P$. This element satisfies the property that $\mathbb E[\varphi_h] = \varphi_h[m] = \langle h, m \rangle$ for all $h \in H$.


$$\operatorname{cov}[\varphi_h|\varphi_{h'}] = \int_X \big\langle h, x - m \big\rangle \big\langle h', x-m \big\rangle \, \mathrm d \mathbf P(x).$$

This is the formula you were looking for. It's just a special case of the usual covariance formula, specialized to the setting where the random variables of interest are continuous linear functionals, and the probability space is a Hilbert space.

  • $\begingroup$ Thank you for your answer! One question about the concept: is the result of the expectation in $R$ or in $X$ or in $X^*$? Is this the same as the mean map or the distribution embedding to RKHS? $\endgroup$ Oct 28, 2015 at 4:00
  • $\begingroup$ I mean, why "the mean can be represented by an element in H"? I don't understand this. "A real number(the mean) can be represented by an element in Hilbert space"? (I mean, by Riesz, functional on H can be represented. why here the mean can also be represented?) $\endgroup$ Oct 28, 2015 at 4:03
  • $\begingroup$ Do you mean that $E[ ]: H^*\to R$ is a functional on $H^*$, so it is in $H^{**}$. And $H^{**}=H$. So then $E[]$, as element in $H^{**}$ be represented by m in $H^*$, so then $E[\phi]=<\phi,m>_{H^*}=\phi(m)=<h_{\phi},m>$? $\endgroup$ Oct 28, 2015 at 4:26
  • $\begingroup$ It seems confusing when you write that every continuous linear functional $\varphi\in X^*$ is a random variable $\varphi:X\to\mathbb R$. $X$ is only a Banach space, it is not a probability space, right? Of course, we can use Borel $\sigma$-algebra, but what is the probability measure of this space? $\endgroup$
    – Cm7F7Bb
    Mar 8, 2016 at 10:14
  • $\begingroup$ Just saw the question. I'd try to explain part(s) of your questions: In order to define a random variable $X:\Omega \to \mathbb{R}$, we don't need a probability measure on $\Omega$, we just need a sigma algebra, which in this case is the Borel one. This is because by definition, a random variable is just a measurable function, i.e. a function that pulls back the open sets of $\mathbb{R}$ to measurable subsets of $\Omega$, i.e. the subsets in the Borel sigma algebra. Next, why is the mean in $H$? Hint: it's not a real number! Think when $H=\mathbb{R}^d$, then $E(X)\in \mathbb{R}^d$ also $\endgroup$ Jun 24, 2016 at 2:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.