1
$\begingroup$

When I was going through a proof, I saw the following step:

$$\frac{1}{p}\operatorname{trace}(\Sigma) \leq \frac{1}{\sqrt{p}} \|\Sigma\|$$

where, $p$ is the number of variables, $\sigma$ is the population covariance matrix and $\|\cdot\|$ stands for Frobenius norm.

They stated that this from jensen's inequality. Can anyone explain me how they got this result.

Thanks alot

$\endgroup$
2
$\begingroup$

It follows from (3) here with $\phi(t)=t^2$ which implies that for $x=[x_1,\ldots,x_p]$, we have $$ \left(\frac{\sum_i x_i}{p}\right)^2\leq \sum_i\frac{x_i^2}{p}. $$ You just need to replace $x$ with the vector of eigenvalues of $\Sigma$ to get the result.

EDIT: Let $\lambda_1,\ldots,\lambda_p$ be the positive eigenvalues of $\Sigma$ (since it is a covariance matrix, I suppose it is symmetric and positive definite; it works also for semi-definite matrices). Then $$ \mathrm{trace}(\Sigma)=\sum_{i=1}^p\lambda_i, \qquad \|\Sigma\|=\left(\sum_{i=1}^p\lambda_i^2\right)^{1/2}. $$ So putting $\lambda$'s instead of $x$'s of the inequality above gives $$ \left(\frac{\sum_i\lambda_i}{p}\right)^2 \leq \sum_i\frac{\lambda_i^2}{p} \quad\Leftrightarrow\quad \frac{1}{p^2}[\mathrm{trace}(\Sigma)]^2\leq\frac{1}{p}\|\Sigma\|^2. $$ Then take the square root.

EDIT: The Jensen's inequality (3) on the linked page states that $$ \phi\left(\frac{\sum_i x_i}{n}\right)\leq \frac{\sum_i \phi(x_i)}{n} $$ for $n$ numbers $x_1,\ldots,x_n$ in the domain of a convex function $\phi$. Just put $\phi(t)=t^2$ (which is convex).

$\endgroup$
  • $\begingroup$ Still I did not get. It would be great if you could explain it. $\endgroup$ – shani Oct 3 '13 at 13:19
  • $\begingroup$ Bit confused with this given formula. How did you get it from Jensen's inequality $\endgroup$ – shani Oct 3 '13 at 13:23
  • $\begingroup$ Ah OK, let me edit again :) $\endgroup$ – Algebraic Pavel Oct 3 '13 at 13:28
  • $\begingroup$ Yes, I got that point.But how you got the formula with x's from Jensen;s inequality? $\endgroup$ – shani Oct 3 '13 at 13:30
  • $\begingroup$ It's in the second edit. $\endgroup$ – Algebraic Pavel Oct 3 '13 at 13:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.