# Inner product and norms for random vectors

From wikipedia inner product page: the expected value of product of two random variables is an inner product $\langle X,Y \rangle = \operatorname{E}(X Y)$. How it can be generalized in case of random vectors?

Or more generally for any probability measure. Let $\mathbb{P}$ be a set of all probability measures defined on $X$, and let $\mathbb{M}$ be the linear span of $\mathbb{P} - \mathbb{P}$. How an inner product can be defined on $\mathbb{M} \times \mathbb{M}$?

I've looked to the norm like $$\|P - Q\|= \sup_{f} \left| \int f \, dP - \int f \, dQ \right|$$ But it seems that this norm doesn't satisfy the parallelogram law (so $\langle x, y\rangle = \frac{1}{4}( \|x + y\|^{2} - \|x - y\|^{2})$ trick cannot be used). Is it possible to proof this?

• I'm not sure I understand the second question or how it's a generalization of the first question. For random vectors in, say, a Hilbert space, I guess you can take the expectation of their inner product. – Qiaochu Yuan May 3 '11 at 20:38
• let me clarify, in first case we assume that we have inner product defined as $\langle X, Y \rangle = \int x y \, dP_{XY}$. In second case i'm more interested possibility to define inner products based on so called integral probability metrics. – TheBug May 3 '11 at 21:10

$\mathbb M$ would be the space of signed measures on $X$ (presumably with respect to a particular $\sigma$-algebra). This is a Banach space with the total-variation norm, but not a Hilbert space, and so it doesn't have a natural inner product.
• Right, every Hilbert space is a Banach space because, but not every Banach space is a Hilbert space. A necessary and sufficient condition for a Banach space to be associated to an inner product (make it into a Hilbert space) is the parallelogram identity. So can you proof that TV norm doesn't satisfy the parallelogram identity. Btw, TV norm is just a special case of the norm defined in question, it depends on choise of set of $f$ functions in supremum. – TheBug May 3 '11 at 21:31
• That the total variation norm doesn't satisfy the parallelogram identity is very easy (try some examples). A bit less obvious is that $\mathbb M$ is not isomorphic to a Hilbert space (except in the trivial case where $X$ consists of finitely many atoms). This can be seen from the fact that $\mathbb M$ has a closed subspace isomorphic to $\ell_1$, and $\ell_1$ is not reflexive. – Robert Israel May 5 '11 at 15:51