how can I prove ${\lambda _n}\left( {{X^T}AX} \right) \leqslant \left\| X \right\|_F^2{\lambda _n}\left( A \right)$ where $A$ is a PD matrix

Question

The following inequality intuitively holds in my opinion, however I am facing hard time proving it

${\lambda _n}\left( {{X^T}AX} \right) \leqslant \left\| X \right\|_F^2{\lambda _n}\left( A \right)$

Keep in mind that $\lambda_n$ is the smallest eigenvalue of a $n$-by-$n$ square matrix. The matrix $A$ is a symmetric, positive definite matrix

All I know from matrix linear algebra is that

${\lambda _n}\left( {{X^T}AX} \right) \leqslant {\lambda _1}\left( {{X^T}AX} \right) = \rho \left( {{X^T}AX} \right) \leqslant {\left\| {{X^T}AX} \right\|_F} \leqslant \left\| X \right\|_F^2{\left\| A \right\|_F}$

where $\rho$ is the spectral radius.

Answer 1

One easy way to prove it is to utilise Sylvester's secular theorem that $ST$ and $TS$ share the same multi-set of nonzero eigenvalues. In particular, $ST$ and $TS$ share the same spectra if both $S$ and $T$ are square matrices. Thus \begin{aligned} \lambda_n(X^TAX) &=\lambda_n\left((X^TA^{1/2})(A^{1/2}X)\right)\quad\text{(here we need $A$ to be PSD)}\\ &=\lambda_n\left((A^{1/2}X)(X^TA^{1/2}))\right)\\ &\le\lambda_n\left(A^{1/2}\left(\|X\|_2^2I\right)A^{1/2})\right) \quad\text{(because $XX^T\preceq \|X\|_2^2I$)}\\ &=\|X\|_2^2\lambda_n(A)\\ &\le\|X\|_F^2\lambda_n(A).\\ \end{aligned} As we see in the above, $\|X\|_2^2\lambda_n(A)$ a sharper upper bound of $\lambda_n(X^TAX)$ than $\|X\|_F^2\lambda_n(A)$.