Deeper Algebraic Structure to Random Vectors?

Question

Given a probability space $\Omega,$ the space of square-integrable measurable functions $\Omega \to \mathbb{R}^n$ ("random vectors") can be made a vector space over $\mathbb{R}$ in a natural way. Call this space $V.$ In probability theory, we proceed to define several operators on this space, like the expectation operator $E : V \to \mathbb{R}^n$ given by $(X_1,X_2...,X_n) \mapsto (E(X_1),E(X_2)...,E(X_n))$.

However, going just a bit deeper into the theory, we start to see some properties of $E$ nicer than linearity over $\mathbb{R}$ would alone suggest. For example, for any $k \times n$ matrix $A$, we find that $E(AX) = AE(X).$ Similar occurrences occur with the bilinear covariance operator $\mathrm{Cov} : V \to \mathbb{R}^{n \times n}$. For example, for any $k \times n$ matrices $A$ and $B,$ we find $\mathrm{Cov}(AX,BY) = A\mathrm{Cov}(X,Y)B^T,$ where $B^T$ denotes the transpose of $B.$

On one level, one can just view this as matrix algebra (and this may be all there is to it). But I've always been inclined to look for deeper algebraic structure than just matrix algebra when I see matrices, so I'm wondering if there's a deeper algebraic reason to this. For example, we could have viewed $V$ as a module over $n \times n$ matrices, but this approach doesn't seem to explain the transposes and the generalization to $k \times n$ matrices with $k \neq n.$ So, I'm wondering if there's some algebraic structure to $V$ in which the "matrix linearity" of the form seen in $E$ and $\mathrm{Cov}$ become natural (and hence easy to remember!).

Answer 1

You're using $A$ to represent two different linear operators: $A_V:V\rightarrow V$ and $A_{\mathbb{R}^n}:\mathbb{R}^n\rightarrow\mathbb{R}^n$. So, what you've really shown is that $E(A_VX) = A_{\mathbb{R}^n}E(X)$.

Perhaps the significance of $A_V$ and $A_{\mathbb{R}^n}$ having the same representation has more to do with the component-wise independence of $E$ on $V$, which is what allows linear operators to commute.

As a counterexample, take another matrix $B$, and define $E^\prime:=E\circ B$. In this case, $AE^\prime(X)\neq E^\prime(AX)$ because in general $AB\neq BA$.

As another counterexample, suppose that $V$ is a space of distributions over infinitely-differentiable functions on $\mathbb{R}$, and that $\frac{d}{dx}$ is our linear operator. We could in theory create some random variable $X$ where $E(X)$ isn't differentiable, and thus $\frac{d}{dx}E(X)\neq E(\frac{d}{dx}X)$.

The overall point being that your example of a linear operator commuting with some $V\rightarrow\mathbb{R}^n$ is a very special case.