# How to define the expectation and covariance of a random variable taking values in an inner product space?

I'm dealing with random variables that takes values in an inner product space, that is possibly finite dimensional but not necessarily Euclidean. To be precise: let $$dim(V)=d$$, and we equip $$V$$ with the topology induced from the norm coming from the inner product $$<,>$$. Once done we define a random variable $$X : \Omega \to V$$ as a function that pulls back the open subsets of $$V$$ on the Borel sigma algebra of $$\Omega$$.

My question is: given the above, how do we go about the definition of $$\mathbb{E}X, \mathbb{Cov} [X]$$ respectively? So to do this, let's take a basis $$B$$ of $$V$$, and then a vector space isomorphism $$\Phi_B: V \to B$$ that takes the basis $$B$$ of $$V$$ onto the canonical basis $$\{v_1,...v_n\}$$ of $$\mathbb{R}^d$$.

Next, we define the the expectation as:

Suggested definition 1: $$\mathbb{E}X:= \Phi_B^{-1} \mathbb{E}[\Phi_B \circ X] \in V$$. Note that we can also try to define it as follows:

Suggested definition 2: Let $$B:=\{v_1,...v_d\}$$ be a basis. Then we express: $$X= \sum_{i=1}^{d}X_iv_i, X_i: \Omega \to \mathbb{R}$$ are real random variables ("the components w.r.t. the basis"). Then we can define: $$\mathbb{E}X:= \sum_{i=1}^{d}(\mathbb{E}X_i) v_i$$. I think this definition is more amenable to prove basis-independence, by the following argument. Assume that: $$X= \sum_{i=1}^{d}X_iv_i = \sum_{i=1}^{d}Y_i w_i$$. So, by taking the integral w.r.t. the prob. measure on $$\Omega$$, and taking the $$v_i$$ out of the integrals as they're ($$V$$-valued) constants, we arrive at: $$\sum_{i=1}^{d}(\mathbb{E}X_i)v_i = \sum_{i=1}^{d}(\mathbb{E}Y_i) w_i$$

But then of course, we need to prove the equivalence between these two suggested definitions, how do we do that?

Next, we define the covariance as follows:

Suggested definition 1: Consider $$\mathbb{Cov}[\Phi_B \circ X]$$, which is a matrix, but think of a linear operator on $$\mathbb{R}^d$$. Then define $$\mathbb{Cov} [X]$$ as a linear operator on $$V$$, defined below:

$$\mathbb{Cov} [X] := \Phi_B^{-1} \circ \mathbb{Cov}[\Phi_B \circ X] \circ \Phi_B$$.

My problem is: I can't readily see why these two definitions above are basis independent?

To prove basis independence for $$\mathbb{E}X$$, we have to prove that for any two bases $$B, B'$$ of $$V$$, we must have:

$$\Phi_B^{-1} \mathbb{E}[\Phi_B \circ X] = \Phi_{B'}^{-1} \mathbb{E}[\Phi_{B'} \circ X]$$. But I'm not sure why it's true?

Similarly, to prove that the definition of covariance is basis-independent, we need to show:

$$\Phi_B^{-1} \circ \mathbb{Cov}[\Phi_B \circ X] \circ \Phi_B = \Phi_{B'}^{-1} \circ \mathbb{Cov}[\Phi_{B'} \circ X] \circ \Phi_{B'}$$. But I'm not sure why it's true?

Suggested definition 2: (only attempted, not finished!) Much like the way I defined the expectation in the suggested definition 2, we can go about defining the covariance using its linearity in one variable: $$\mathbb{Cov}(X):= \mathbb{Cov}(X,X) = \mathbb{Cov}(\sum_{i=1}^{d}X_i v_i,\sum_{j=1}^{d}X_j v_j) = \sum_{i=1}^{d} \sum_{j=1}^{d}\mathbb{Cov}(X_i, X_j)???$$ I put $$???$$ here because I don't know what should be there, but for sure $$???$$ should be a linear operator on $$V$$. If you take $$V:= \mathbb{R}^d$$ with its canonical inner product, then $$???$$ will be the matrix/linear operator $$e_i e_j^{T}$$. I'll think more about it!

Any constructive help would be appreciated!!!

• For multidimensional random variables, the expected value is defined component wise. See, e.g. en.m.wikipedia.org/wiki/Expected_value the end of “general case” subsection under “definition”. – Nap D. Lover Feb 15 at 0:43
• @NapD.Lover Thanks but not sure how that helps? I already know how to define expectation for multidimensional random variables. The question is how to define expectaton for general vector space valued random variables. The ptoblem is with the basis. – Mathmath Feb 15 at 1:49
• I haven't worked out the details, but you probably can use linearity of expectation and express everything with respect to two different bases. – rubikscube09 Feb 15 at 7:01
• This is a nice question. I think the basis independence of the expectation should easily follow by using that $A \int f dP = \int A f dP$. For the covariance, I think there is a way to define it in a basis independent way, as a bilinear form on $V'$ (the dual space of $V$). Precisely, as $\mathrm{Cov}(X) \langle \varphi, \psi\rangle= \Bbb{E}[\varphi(X)\psi(X)]$. This shows that the "usual" definition "abuses" the identification of $\Bbb{R}^d$ with it's dual, by virtue of the standard scalar product. – PhoemueX Feb 15 at 13:26
• @PhoemueX Thanks for your comment! I changed the question slightly, where I replaced "vector space" by "inner product space". The spaces are still finite dimensional, at least for the moment. So now, a bilinear form on $V'$ is basically a lineart operator $V$ by canonical (=basis independent) duality between the inner product spaces. If you'eve a reference or if you can do the calculation by yourself that checks all the basis independence, I'd appreciate if you could write an answer. In the meantime, I'm adding something to the expectation part. Do you mind writing an answer if – Mathmath Feb 15 at 19:23