1
$\begingroup$

I'm looking at a simplified case of this question where I have a random vector variable $X$ in dimension $k$. I know this about $X$: its mean $E[X]$, its covariance matrix, the mean of its $L^2$ norm $E[{||X||]}$, the variance of its $L^2$ norm $\operatorname{Var}(||X||)$. I have another random vector variable $Y$ in the same space, and I know that $\overline{||X - Y||} = A$ where $A$ is some constant, $\operatorname{Var}(||Y||)$ = $\operatorname{Var}(||X||)$ and $X$ and $Y$ are independent.

I'm trying to figure out how $\operatorname{Var}(||X - Y||)$ relates to the properties I know of $X$ and $Y$, either through a strict equality or an upper bound. Here's my attempt at finding an upper bound:

$$ \begin{align} \operatorname{Var}(||X - Y||) & = E[(||X - Y|| - E[||X - Y||])^2] \\ & = E[(||X - Y|| - A)^2] \\ & = E[||X - Y||^2] - 2 A \times E[||X - Y||] + A^2 \\ & = E[||X - Y||^2] - 2A^2 + A^2 \\ & = E[||X - Y||^2] - A^2 \\ & \leq E[(||X|| + ||Y||)^2] - A^2 \\ & \leq E[||X||^2] + E[||X||||Y||] + E[||Y||)^2] - A^2 \end{align} $$

But I'm stopped at the second order moment. Plus, I think the $A$ constant should somehow disappear. I'm conjecturing that

  1. $\operatorname{Var}(||X - Y||) \leq \operatorname{Var}(||X||) + \operatorname{Var}(||Y||) = 2 \operatorname{Var}(||X||)$
  2. The equality is strict if $X$ and $Y$ are independent.

I'm also wondering if there is a formula in the more general case.

$\endgroup$
0

1 Answer 1

1
$\begingroup$

Counter-example

Take $X, Y$ i.i.d. with $P[X=-1]=P[X=1]=1/2$. Then $Var(|X|)+Var(|Y|)=0$ but $Var(|X-Y|)>0$.

A modified statement:

Let $X=(X_1, ..., X_n)$ and $Y=(Y_1,...,Y_n)$ be random vectors in $\mathbb{R}^n$. Assume $E[X_i^2]$ and $E[Y_i^2]$ are finite for all $i \in \{1, ..., n\}$, and that $X_i$ and $Y_i$ are uncorrelated for each $i \in \{1, ..., n\}$. Let $||x||= \sqrt{\sum_{i=1}^n x_i^2}$ denote the Euclidean norm. Then \begin{align*} Var(||X-Y||) &\leq Var(||X||) + Var(||Y||) \\ & \quad + (E[||X||]^2-||E[X]||^2) + (E[||Y||]^2 - ||E[Y]||^2) \end{align*}

Proof:

We have \begin{align*} &||X-Y||^2 = ||X||^2 + ||Y||^2 -2\sum_{i=1}^n X_iY_i\\ \implies & E[||X-Y||^2] = E[||X||^2] + E[||Y||^2] -2\sum_{i=1}^n E[X_i]E[Y_i] \quad (Eq. *) \end{align*} where we have used the fact that $X_i$ and $Y_i$ are uncorrelated for each $i\in \{1, \ldots, n\}$.

On the other hand, by Jensen's inequality with the convex function $||\cdot||$ we have $$ E[||X-Y||] \geq ||E[X]-E[Y]|| \geq 0$$ and since $a\geq b \geq 0 \implies a^2 \geq b^2$ for any real numbers $a,b$, we have
\begin{align} E[||X-Y||]^2 &\geq ||E[X]-E[Y]||^2 \\ &= ||E[X]||^2 + ||E[Y]||^2 - 2\sum_{i=1}^n E[X_i]E[Y_i] \quad (Eq. **) \end{align} Thus \begin{align*} Var(||X-Y||) &= E[||X-Y||^2] - E[||X-Y||]^2\\ &\leq E[||X||^2] - ||E[X]||^2 + E[||Y||^2] - ||E[Y]||^2 \end{align*} where the final inequality combines (Eq. *) and (Eq. **). $\Box$

$\endgroup$
3
  • $\begingroup$ Thank you! Are there any "common" classes for X where E[||X||]^2 = ||E[X]||^2? I'm wondering for example, if X is Gaussian this may be the case $\endgroup$
    – Flavian
    Commented Jul 2, 2020 at 20:14
  • $\begingroup$ After joting down notes, E[||X||]^2 - ||E[X]||^2 = Sum_i(Var[X_i]). So they would be equal if every Var(X_i) = 0 ie X always takes the same value $\endgroup$
    – Flavian
    Commented Jul 2, 2020 at 20:28
  • $\begingroup$ I cannot get your variance equality. However since $f:\mathbb{R}^n\rightarrow\mathbb{R}$ given by $f(x) = ||x||$ is a strictly convex function, it holds that $E[f(X)]\geq f(E[X])$ with equality if and only if $P[X=E[X]]=1$. $\endgroup$
    – Michael
    Commented Jul 2, 2020 at 21:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .