Let $X \in \mathbb{R}^{n\times p}$ be a data matrix where $n \geq p$.

Say that I know that the covariance matrix $$X^T X$$ has $n$ real eigenvalues $\lambda_1,\ldots,\lambda_n$.

Is it possible to produce a lower bound and/or upper bound on the eigenvalues of a scaled covariance matrix:

$$X^T D X$$

Here $D$ is a $n\times n$ diagonal "scaling" matrix with $d_1,\ldots,d_n$ on the diagonal. The elements $d_1,\ldots,d_n$ are real, positive, and bounded in the interval $[d_{min},d_{max}]$ where $0 < d_{min} < d_{max} < 1$.


Upper Bound:

Let $\rho$ denote the maximal eigenvalue of $X^TX$. Let $\|A\|$ denote the spectral norm of the matrix $A$, given by $\|A\| = \lambda_{max}(\sqrt{A^TA})$. We then have $$ \|X^TDX\| \leq \|X^T\|\cdot \|D\| \cdot \|X\| $$ On the other hand, we have $$ \|X\| = \sqrt{\rho}\\ \|X^T\| = \lambda_{max}(\sqrt{XX^T}) = \lambda_{max}(\sqrt{X^TX}) = \sqrt{\rho}\\ \|D\| = \lambda_{max}(D) \leq 1 $$ So, all together, we have $$ \lambda_{max}(X^TDX) \leq \|X^TDX\| \leq \|X^T\|\cdot \|D\| \cdot \|X\| \leq \rho $$

Lower Bound:

First of all, note that $X^TDX$ is positive semidefinite, so its eigenvalues are certainly non-negative.

Let $d = d_{min}$. We note that the matrix $$ X^TDX - d(X^TX) = X^T(D - dI)X $$ is necessarily positive definite. It follows that $$ \lambda_{min}(X^TDX) \geq \lambda_{min}(dX^TX) = d\lambda_{min}(X^TX) $$ In fact, we could also have used this trick for the upper bound by noting that $X^T(\rho I - D)X$ is positive semidefinite.

  • $\begingroup$ Thanks! I see why $X^T X$ and $X^T X D$ are positive definite, but I don't see why $X^T X D X$ is positive definite. Was that just a typo? $\endgroup$ – Elements Jun 15 '15 at 21:52
  • $\begingroup$ Yes, that was a typo. Also, $X^TDX$, not $X^TXD$. $\endgroup$ – Omnomnomnom Jun 15 '15 at 21:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.