Bounding the Eigenvalues of a Scaled Covariance Matrix

Let $X \in \mathbb{R}^{n\times p}$ be a data matrix where $n \geq p$.

Say that I know that the covariance matrix $$X^T X$$ has $n$ real eigenvalues $\lambda_1,\ldots,\lambda_n$.

Is it possible to produce a lower bound and/or upper bound on the eigenvalues of a scaled covariance matrix:

$$X^T D X$$

Here $D$ is a $n\times n$ diagonal "scaling" matrix with $d_1,\ldots,d_n$ on the diagonal. The elements $d_1,\ldots,d_n$ are real, positive, and bounded in the interval $[d_{min},d_{max}]$ where $0 < d_{min} < d_{max} < 1$.

1 Answer

Upper Bound:

Let $\rho$ denote the maximal eigenvalue of $X^TX$. Let $\|A\|$ denote the spectral norm of the matrix $A$, given by $\|A\| = \lambda_{max}(\sqrt{A^TA})$. We then have $$\|X^TDX\| \leq \|X^T\|\cdot \|D\| \cdot \|X\|$$ On the other hand, we have $$\|X\| = \sqrt{\rho}\\ \|X^T\| = \lambda_{max}(\sqrt{XX^T}) = \lambda_{max}(\sqrt{X^TX}) = \sqrt{\rho}\\ \|D\| = \lambda_{max}(D) \leq 1$$ So, all together, we have $$\lambda_{max}(X^TDX) \leq \|X^TDX\| \leq \|X^T\|\cdot \|D\| \cdot \|X\| \leq \rho$$

Lower Bound:

First of all, note that $X^TDX$ is positive semidefinite, so its eigenvalues are certainly non-negative.

Let $d = d_{min}$. We note that the matrix $$X^TDX - d(X^TX) = X^T(D - dI)X$$ is necessarily positive definite. It follows that $$\lambda_{min}(X^TDX) \geq \lambda_{min}(dX^TX) = d\lambda_{min}(X^TX)$$ In fact, we could also have used this trick for the upper bound by noting that $X^T(\rho I - D)X$ is positive semidefinite.

• Thanks! I see why $X^T X$ and $X^T X D$ are positive definite, but I don't see why $X^T X D X$ is positive definite. Was that just a typo? – Elements Jun 15 '15 at 21:52
• Yes, that was a typo. Also, $X^TDX$, not $X^TXD$. – Ben Grossmann Jun 15 '15 at 21:54