2
$\begingroup$

Consider random vectors $X,Y\in \mathbb{R}^d$ on a unit-euclidean-ball $\big($i.e, $||X||_2=||Y||_2=1\big)$, and a matrix $M \in \mathbb{R}^{d\times d}$. How to calculate the following expectation: $$\mathbb{E}\big[\big|X^TMY\big|\big]$$

Edit: Added the modulus sign

Thanks in advance!

$\endgroup$

closed as off-topic by Did, John B, Xander Henderson, José Carlos Santos, user223391 Jul 29 '18 at 22:00

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question is missing context or other details: Please improve the question by providing additional context, which ideally includes your thoughts on the problem and any attempts you have made to solve it. This information helps others identify where you have difficulties and helps them write answers appropriate to your experience level." – Did, John B, Xander Henderson, José Carlos Santos, Community
If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ What have you done so far? $\endgroup$ – Jean-François Gagnon Sep 8 '17 at 22:18
  • $\begingroup$ Vectors are uniformly drawn from the surface of the ball? $\endgroup$ – Brian Tung Sep 8 '17 at 22:36
  • $\begingroup$ @Brian Tung: Yes, by default, that's the assumption. $\endgroup$ – quasi Sep 8 '17 at 22:40
3
$\begingroup$

I have a proof for a lower bound on that value, but I'm not sure at the moment if it's possible to show a tight upper bound too.

Let me start with a few facts:

  • We can write the random vectors $x, y \in \mathbb{R}^d$ as $x=u/\lVert u \rVert, y=v/\lVert v \rVert$, with $u, v \sim \mathcal{N}(0, \textrm{Id}_d)$. See, e.g., Proof that normalized vector of Gaussian variables is uniformly distributed on the sphere.

  • $u \sim \mathcal{N}(0, \textrm{Id}_d) \implies Au \sim \mathcal{N}(0, AA^T)$ for any matrix $A$; in particular, for $A$ orthogonal, $Au \sim \mathcal{N}(0, \textrm{Id}_d)$.

  • Orthogonal transforms preserve norms induced by inner-products (by definition): $\lVert u \rVert = \langle u, u \rangle^{1/2} = \langle Au, Au \rangle^{1/2} = \lVert Au \rVert$.

  • $M = UDV^T$, a singular value decomposition of $M$.

Thus, from the first three facts we have that $w = U^T x$ and $z = V^T y$ are uniformly distributed on the unit sphere as well. Then, $w^T D z = x^T U D V^T y$ = $x^T M y$, and so we can study the simplified expression $\mathbb{E}\ \lvert x^T D y \rvert$ where $D$ is a diagonal matrix. To obtain a lower bound, proceed as follows

\begin{align} \mathbb{E}\ \lvert x^T D y \rvert &= \mathbb{E}\ \lvert \textrm{Tr}\ x^T D y \rvert &&\text{trace trick} \\ &= \mathbb{E}\ \lvert \textrm{Tr}\ y x^T D \rvert &&\text{trace cyclicity} \\ &\ge \lvert \mathbb{E}\ \textrm{Tr}\ y x^T D \rvert &&\text{Jensen's inequality} \\ &= \lvert \textrm{Tr}\ \mathbb{E} \big[ y x^T \big] D \rvert &&\text{trace linearity} \\ &= \lvert \textrm{Tr}\ \textrm{Id}_d D \rvert &&\text{$x, y \stackrel{\textrm{iid}}{\sim} \mathcal{N}(0, \textrm{Id}_d)$} \\ &= \sum_{i=1}^d \sigma_i(M), &&\text{non-negativity of singular values} \end{align} where $\sigma_i(M)$ is the $i$th largest singular value of $M$.


I think it's also possible to show an upper bound in terms of the absolute value of the sum of the singular values of $M$, as follows. Let $y x^T = U \Sigma V^T$ be a SVD of the random matrix $y x^T$ (note that $U$ and $V$ are different than before, but since I didn't use those ones explicitly, let me re-use their names). Then,

\begin{align} \mathbb{E}\ \lvert x^T D y \rvert &= \mathbb{E}\ \lvert \textrm{Tr}\ U \Sigma V^T D \rvert \\ &= \mathbb{E}\ \Big\lvert \sum_{i=1}^d \sigma_i(M) u_i^T \Sigma v_i \Big\rvert &&\text{trace definition} \\ &\le \mathbb{E}\ \Big\lvert \max\limits_{\lVert u \rVert = \lVert v \rVert = 1} \sum_{i=1}^d \sigma_i(M) u^T \Sigma v \Big\rvert &&\text{upper bound} \\ &= \mathbb{E}\ \Big\lvert \sum_{i=1}^d \sigma_i(M) \sigma_1(y x^T) \Big\rvert &&\text{maximum singular value} \\ &= \Big( \sum_{i=1}^d \sigma_i(M) \Big) \cdot \mathbb{E}\ \big\lvert \sigma_1(y x^T) \big\rvert, \end{align}

but there is this additional factor, the expected value of the (absolute value of the) maximum singular value of the outer product of two standard Gaussian vectors, which I'm not sure how to bound. I am familiar with a result which says that $$\lvert 2 - \mathbb{E}\ \lVert W \rVert \rvert \le o(1),\quad \textrm{for}\ W \sim \mathcal{N}(0, 1)^{n \times n},$$ where $\lVert \cdot \rVert$ denotes the spectral norm (equal to $\sigma_1$), but notice that $y x^T$ does not (necessarily) have normally-distributed entries (see, e.g., Is the product of two Gaussian random variables also a Gaussian?).

$\endgroup$
1
$\begingroup$

Assume $X,Y$ are independent random vectors, chosen uniformly from the standard unit $d$-sphere.

Fix $Y$, and let $Z=MY$.

Then $Z$ is just some fixed vector in $\mathbb{R}^d$.

By symmetry, for $X$ chosen uniformly at random from the standard unit $d$-sphere, $$E[X\cdot Z] = E[(-X)\cdot Z] = E[-(X\cdot Z)]= -E[X\cdot Z]$$ hence $E[X\cdot Z]=0$.

Therefore $E[X^TMY]=0$.

$\endgroup$
  • $\begingroup$ Thanks for the solution! I have modified the question to reflect the quantity I wanted to measure. $\endgroup$ – Vivek Bagaria Sep 10 '17 at 20:06

Not the answer you're looking for? Browse other questions tagged or ask your own question.