The question goes as:

Let $\sigma_1(A) \geq \sigma_2(A) \geq \cdots \geq \sigma_r(A)$ be all (non-zero) singular values of an order $n$ real matrix $A$. Prove: \begin{align*} \sigma_k(A) = \sup_{\substack{X \in \mathbb{R}^{k \times n}\\ X \neq 0}}\frac{\sigma_k(XA)}{\sigma_1(X)}, \quad \sigma_{n + 1 - k}(A) = \inf_{\substack{X \in \mathbb{R}^{k \times n} \\ X \neq 0}}\frac{\sigma_1(XA)}{\sigma_k(X)}, \quad \forall k = 1, 2, \ldots, r. \end{align*}

Take the first equality as example, by the definition of singular values and the representation of eigenvalues, I am able tho show that $\sigma_k(A) \leq \sup_{X \in \mathbb{R}^{k \times n}}\frac{\sigma_k(XA)}{\sigma_1(X)}$. It seems difficult to obtain the reverse direction. In particular, I don't know how to use the condition that $A$ is symmetric.

  • $\begingroup$ Taking into account that the singular values of $A$ are the square roots of the eigenvalues of $A^*A$, the min-max theorem (en.wikipedia.org/wiki/Min-max_theorem) should help. $\endgroup$
    – amsmath
    Feb 28, 2021 at 3:39
  • $\begingroup$ @amsmath I tried this way from the very beginning but didn't reach the destination. $\endgroup$
    – Zhanxiong
    Feb 28, 2021 at 3:45
  • $\begingroup$ $$ \left(\sup_{X\in\mathbb R^{1\times n}}\frac{\sigma_1(XA)}{\sigma_1(X)}\right)^2 = \sup_{X\in\mathbb R^{1\times n}}\frac{\lambda_1(XAA^*X^*)}{\lambda_1(XX^*)} = \sup_{x\in\mathbb R^{n}}\frac{x^*AA^*x}{x^*x} = \lambda_1(AA^*) = \sigma_1(A)^2. $$ $\endgroup$
    – amsmath
    Feb 28, 2021 at 3:55
  • $\begingroup$ how about the general $k > 1$ case? $\endgroup$
    – Zhanxiong
    Feb 28, 2021 at 4:31
  • 2
    $\begingroup$ @Zhanxiong The inequality which you haven't proven is actually the easy one, in that it holds for all $X,A$ not just $A$ real symmetric. Note that$$A^*X^*XA \le A^*\Vert X^* X\Vert A = \Vert X \Vert^2 A^* A = \sigma_1(X)^2 A^*A.$$ Consequently, $\sigma_k(XA) \le \sigma_1(X) \sigma_k(A)$ for all $k$. $\endgroup$
    – J. Loreaux
    Mar 2, 2021 at 23:23

1 Answer 1


$\newcommand{\real}{\mathbb{R}}$ $\newcommand{\diag}{\mathrm{diag}}$

I will expand @J.Loreaux excellent comment to a more detailed answer for future references. Also, I will take @user8675309's suggestion to allow the singular values to be zero.

For any $1 \leq k \leq r$ and $X \in \real^{k \times n}$, suppose $$X^TX = O\diag(\sigma_1^2(X), \ldots, \sigma_n^2(X))O^T.$$ is the spectral decomposition of $X^TX$, where $O$ is an order $n$ orthogonal matrix. Recall the notation: for two symmetric matrices $A$ and $B$, $A - B \geq 0$ means the matrix $A - B$ is positive semi-definite. It then follows that

\begin{align*} & \Delta_k := \sigma_1^2(X)A^TA - (XA)^T(XA) = \sigma_1^2(X)A^TA - A^TX^TXA \\ = & A^T(\sigma_1^2(X)I_{(n)} - X^TX)A = A^TO\diag(0, \sigma_1^2(X) - \sigma_2^2(X), \ldots, \sigma_1^2(X) - \sigma_n^2(X))O^TA \geq 0. \end{align*}

Together with $A^TX^TXA \geq 0$, by Courant-Fischer Theorem (see this link), we then have for every $1 \leq k \leq r$: \begin{align*} \lambda_k(\sigma_1^2(X)A^TA) = \lambda_k(A^TX^TXA + \Delta_k) \geq \lambda_k(A^TX^TXA), \end{align*} which implies \begin{align*} \sigma_1(X)\sigma_k(A) \geq \sigma_k(XA). \end{align*} This shows \begin{align*} \sigma_k(A) \geq \sup_{\substack{X \in \real^{k \times n} \\ X \neq 0}}\frac{\sigma_k(XA)}{\sigma_1(X)}. \end{align*} The proof to the other inequality is analogous.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .