# How do random variables form vector spaces with a defined inner product?

I am well over my head on this topic, but I'm asking for a couple of conceptual, framing statements to understand the broader picture at an intuitive level.

In Hilbert Space Methods in Probability and Statistical Inference by Christopher G. Small, Don L. McLeish it can be read:

Let $\mathbf G$ be the set of all functions ${\bf x}: \Omega \to > \mathbb R$ such that $({\mathbf x \land n{\bf 1}})\lor(-n{\bf1})\in > {\mathbf H}$ for all natural numbers $n$. The set $\mathbf G$ can be shown to be a vector space under the usual pointwise addition and scalar multiplication.

So what is the connection between the random variables (and specifically their probability density functions) and vector or inner product spaces. Specifically, and if these statements / questions are remotely close to reality, in what way are pdf's linear with scalar multiplication?

In answer to the comment, this is the passage quoted: And this is the link provided by @symplectomorphic, which really resolves the question:

Many of the concepts in this chapter have elegant interpretations if we think of real-valued random variables as vectors in a vector space. In particular, variance and higher moments are related to the concept of norm and distance, while covariance is related to inner product. These connections can help unify and illuminate some of the ideas in the chapter from a different point of view. Of course, real-valued random variables are simply measurable, real-valued functions defined on the sample space, so much of the discussion in this section is a special case of our discussion of function spaces in the chapter on Distributions, but recast in the notation of probability.

As usual, our starting point is a random experiment modeled by a probability space (Ω,F,P). Thus, Ω is the sample space, F is the σ-algebra of events, and P is the probability measure. Our basic vector space V consists of all real-valued random variables defined on (Ω, F, P). Recall that random variables X1 and X2 are equivalent if P(X1 = X2) = 1, in which case we write X1 ≡ X2. We consider two such random variables as the same vector, so that technically, our vector space consists of equivalence classes under this equivalence relation. The addition operator corresponds to the usual addition of two real-valued random variables, and the operation of scalar multiplication corresponds to the usual multiplication of a real-valued random variable by a real (non-random) number. These operations are compatible with the equivalence relation in the sense that if X1 ≡ X2 and Y1≡Y2 then X1 + Y1 ≡ X2 + Y2 and cX1 ≡ cX2 for c ∈ R. In short, the vector space V is well-defined.

• I suspect that $\land$ and $\lor$ are the minimum and the maximum, but what are $n\mathbf1$, $>\Bbb R$ and $>\mathbf H$? Dec 15 '16 at 14:49
• @AlexM. Same question, though I suppose $\mathbf 1$ is just a vector of $1$'s. Dec 15 '16 at 14:51
• Guys, not sure about this, but it may be an indicator variable, the way it is bolded (?). Dec 15 '16 at 14:52
• My comment is not an answer: it is a request that you make the title not prima facie absurd. What you mean to ask is whether, or how, random variables form a vector space, not whether random variables are vector spaces. Dec 15 '16 at 15:06
• Here is a more elementary overview of the basic framework. You seem to be confusing random variables with their densities. The vector space being studied in the book you cite is the space of random variables, aka the space of measurable functions (satisfying certain conditions), not the space of density functions for those random variables. You're right that densities aren't closed under pointwise addition and scalar multiplication. Dec 15 '16 at 15:22

You are mistaking "pointwise scalar multiplication" for "scalar product". For $\alpha \in \Bbb R$ and $\mathbf x \in \mathbf G$ one may define the poinwise scalar multiplication by $(\alpha \mathbf x) (\omega) = \alpha \big( \mathbf x (\omega) \big)$, for $\omega \in \Omega$. There is no mention of scalar product here.
Of course, if you really wanted, you could also define a scalar product by $\langle \mathbf x, \mathbf y \rangle = \int _\Omega \mathbf x (\omega) \mathbf y (\omega) \ \Bbb d \mu$ where $\mu$ is the measure on $\Omega$, but care must be taken in order to assure the convergence of the above integral for all $\mathbf x$ and $\mathbf y$, and in order to make this scalar product non-degenerate (you'll have to work with classes of functions etc.). This could be done, for instance, by requiring $\mu$ to be finite. In any case, this is not what your text tries to convey.
• Doesn't the fact that we're working in a probability space guarantee that $\mu$ is finite? Dec 15 '16 at 14:59
• @TheoreticalEconomist: It does, but is it given that $\Omega$ is a probability space and not just a space with measure? Dec 15 '16 at 15:00