Given a probability space $\Omega,$ the space of square-integrable measurable functions $\Omega \to \mathbb{R}^n$ ("random vectors") can be made a vector space over $\mathbb{R}$ in a natural way. Call this space $V.$ In probability theory, we proceed to define several operators on this space, like the expectation operator $E : V \to \mathbb{R}^n$ given by $(X_1,X_2...,X_n) \mapsto (E(X_1),E(X_2)...,E(X_n))$.

However, going just a bit deeper into the theory, we start to see some properties of $E$ nicer than linearity over $\mathbb{R}$ would alone suggest. For example, for any $k \times n$ matrix $A$, we find that $E(AX) = AE(X).$ Similar occurrences occur with the bilinear covariance operator $\mathrm{Cov} : V \to \mathbb{R}^{n \times n}$. For example, for any $k \times n$ matrices $A$ and $B,$ we find $\mathrm{Cov}(AX,BY) = A\mathrm{Cov}(X,Y)B^T,$ where $B^T$ denotes the transpose of $B.$

On one level, one can just view this as matrix algebra (and this may be all there is to it). But I've always been inclined to look for deeper algebraic structure than just matrix algebra when I see matrices, so I'm wondering if there's a deeper algebraic reason to this. For example, we could have viewed $V$ as a module over $n \times n$ matrices, but this approach doesn't seem to explain the transposes and the generalization to $k \times n$ matrices with $k \neq n.$ So, I'm wondering if there's some algebraic structure to $V$ in which the "matrix linearity" of the form seen in $E$ and $\mathrm{Cov}$ become natural (and hence easy to remember!).

  • $\begingroup$ When you take $A\in M_{k\times n}$ and claim the equality above for $E$, you are assuming that $E$ is defined for all $k$-random vector, right? $\endgroup$ Oct 23, 2020 at 6:45
  • 1
    $\begingroup$ Yes; I guess I've slimily snuck in two notions of $E$ (one with domain on $n-$ random vectors and another on $k-$random vectors, but defined analogously). $\endgroup$
    – Arjun Puri
    Oct 23, 2020 at 6:49
  • $\begingroup$ I ask because then a non-square matrix would move you to a different module. $\endgroup$ Oct 23, 2020 at 6:53
  • $\begingroup$ A more advanced point of view would take a vector space $U$ and consider random vectors to be measurable functions $\Omega \to U$. Then define $E$, but not in terms of its components. (This setting is fine when $U$ is an infinite-dimensional topological vector space.) After that, for your question you would consider linear transformations rather than matrices. $\endgroup$
    – GEdgar
    Oct 30, 2020 at 13:48

1 Answer 1


You're using $A$ to represent two different linear operators: $A_V:V\rightarrow V$ and $A_{\mathbb{R}^n}:\mathbb{R}^n\rightarrow\mathbb{R}^n$. So, what you've really shown is that $E(A_VX) = A_{\mathbb{R}^n}E(X)$.

Perhaps the significance of $A_V$ and $A_{\mathbb{R}^n}$ having the same representation has more to do with the component-wise independence of $E$ on $V$, which is what allows linear operators to commute.

As a counterexample, take another matrix $B$, and define $E^\prime:=E\circ B$. In this case, $AE^\prime(X)\neq E^\prime(AX)$ because in general $AB\neq BA$.

As another counterexample, suppose that $V$ is a space of distributions over infinitely-differentiable functions on $\mathbb{R}$, and that $\frac{d}{dx}$ is our linear operator. We could in theory create some random variable $X$ where $E(X)$ isn't differentiable, and thus $\frac{d}{dx}E(X)\neq E(\frac{d}{dx}X)$.

The overall point being that your example of a linear operator commuting with some $V\rightarrow\mathbb{R}^n$ is a very special case.

  • $\begingroup$ Just as a tangent to my own response, here's a random variable that yields infinitely-differentiable functions but has an expected value that isn't differentiable: $\left\{f_n(x)=\cos(25^n\pi x)\middle\vert P(f_n)=0.5^n\right\}_{n=1}^\infty$. The expected value is the Weierstrass function, which is nowhere differentiable. $\endgroup$ Oct 23, 2020 at 15:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.