Is the variance concave?

Question

Let $X$ be a discrete random variables with values in the set $\{x_1,\ldots, x_n\}\subset\mathbb{R}$. Denote by $p_i$ the probability that $X=x_i$. We can then regard the variance $\mbox{Var}(X)$ as a function of the vector $p\in \Delta^{n-1}\subseteq\mathbb{R}^n$.

Will it be a concave function of $p$? With $n=2$, we get

$$\mbox{Var} (X)=p_1p_2(x_1-x_2)^2=p_1(1-p_1)(x_1-x_2)^2$$

which is concave. I'm not sure how to generalize this beyond two dimensions though.

Answer 1

We have $\mathrm{var}_p\left(X\right)=\mathbb{E}_p\left[X^2\right]-\mathbb{E}_p\left[X\right]^2$. Furthermore, for $\lambda\in\left[0,1\right]$ we can write $$\mathbb{E}_{\lambda p+(1-\lambda)q}\left[X^2\right]=\lambda\mathbb{E}_p\left[X^2\right]+(1-\lambda)\mathbb{E}_q\left[X^2\right].$$ Convexity of $\left(\cdot\right)^2$ and Jensen's inequality then yield $$\lambda \mathbb{E}_p\left[X\right]^2+(1-\lambda) \mathbb{E}_q\left[X\right]^2\geq \left(\lambda\mathbb{E}_p\left[X\right]+(1-\lambda)\mathbb{E}_q\left[X\right]\right)^2=\mathbb{E}_{\lambda p+(1-\lambda)q}\left[X\right]^2.$$

Thus we deduce $$\mathrm{var}_{\lambda p+(1-\lambda)q}\left(X\right)=\mathbb{E}_{\lambda p+(1-\lambda)q}\left[X^2\right]-\mathbb{E}_{\lambda p+(1-\lambda)q}\left[X\right]^2\\\geq\lambda\mathrm{var}_p\left(X\right)+(1-\lambda)\mathrm{var}_q\left(X\right),$$ that means the variance is concave.

Answer 2

We look at the Hessian and see it's negative semi-definite because

i) function itself

\begin{align*} f(p) &= \text{Var}(X)\\ &= E[X^2] - E[X]^2\\ &= p^Ty - (p^Tx)^2, \end{align*} where $y = [x_1^2,\,\ldots,\,x_n^2]^T$.

ii) its gradient and Hessian

\begin{align*} \nabla f &= y - 2(p^Tx)x\\ \nabla f^2 &= -2xx^T. \end{align*} $\nabla f^2$ is negative semi-definite for all $p$ $\Leftrightarrow$ $f$ is concave.