# A Quotient Representation of Singular Values of Symmetric Matrix

The question goes as:

Let $$\sigma_1(A) \geq \sigma_2(A) \geq \cdots \geq \sigma_r(A)$$ be all (non-zero) singular values of an order $$n$$ real matrix $$A$$. Prove: \begin{align*} \sigma_k(A) = \sup_{\substack{X \in \mathbb{R}^{k \times n}\\ X \neq 0}}\frac{\sigma_k(XA)}{\sigma_1(X)}, \quad \sigma_{n + 1 - k}(A) = \inf_{\substack{X \in \mathbb{R}^{k \times n} \\ X \neq 0}}\frac{\sigma_1(XA)}{\sigma_k(X)}, \quad \forall k = 1, 2, \ldots, r. \end{align*}

Take the first equality as example, by the definition of singular values and the representation of eigenvalues, I am able tho show that $$\sigma_k(A) \leq \sup_{X \in \mathbb{R}^{k \times n}}\frac{\sigma_k(XA)}{\sigma_1(X)}$$. It seems difficult to obtain the reverse direction. In particular, I don't know how to use the condition that $$A$$ is symmetric.

• Taking into account that the singular values of $A$ are the square roots of the eigenvalues of $A^*A$, the min-max theorem (en.wikipedia.org/wiki/Min-max_theorem) should help. Feb 28, 2021 at 3:39
• @amsmath I tried this way from the very beginning but didn't reach the destination. Feb 28, 2021 at 3:45
• $$\left(\sup_{X\in\mathbb R^{1\times n}}\frac{\sigma_1(XA)}{\sigma_1(X)}\right)^2 = \sup_{X\in\mathbb R^{1\times n}}\frac{\lambda_1(XAA^*X^*)}{\lambda_1(XX^*)} = \sup_{x\in\mathbb R^{n}}\frac{x^*AA^*x}{x^*x} = \lambda_1(AA^*) = \sigma_1(A)^2.$$ Feb 28, 2021 at 3:55
• how about the general $k > 1$ case? Feb 28, 2021 at 4:31
• @Zhanxiong The inequality which you haven't proven is actually the easy one, in that it holds for all $X,A$ not just $A$ real symmetric. Note that$$A^*X^*XA \le A^*\Vert X^* X\Vert A = \Vert X \Vert^2 A^* A = \sigma_1(X)^2 A^*A.$$ Consequently, $\sigma_k(XA) \le \sigma_1(X) \sigma_k(A)$ for all $k$. Mar 2, 2021 at 23:23

$$\newcommand{\real}{\mathbb{R}}$$ $$\newcommand{\diag}{\mathrm{diag}}$$
For any $$1 \leq k \leq r$$ and $$X \in \real^{k \times n}$$, suppose $$X^TX = O\diag(\sigma_1^2(X), \ldots, \sigma_n^2(X))O^T.$$ is the spectral decomposition of $$X^TX$$, where $$O$$ is an order $$n$$ orthogonal matrix. Recall the notation: for two symmetric matrices $$A$$ and $$B$$, $$A - B \geq 0$$ means the matrix $$A - B$$ is positive semi-definite. It then follows that
\begin{align*} & \Delta_k := \sigma_1^2(X)A^TA - (XA)^T(XA) = \sigma_1^2(X)A^TA - A^TX^TXA \\ = & A^T(\sigma_1^2(X)I_{(n)} - X^TX)A = A^TO\diag(0, \sigma_1^2(X) - \sigma_2^2(X), \ldots, \sigma_1^2(X) - \sigma_n^2(X))O^TA \geq 0. \end{align*}
Together with $$A^TX^TXA \geq 0$$, by Courant-Fischer Theorem (see this link), we then have for every $$1 \leq k \leq r$$: \begin{align*} \lambda_k(\sigma_1^2(X)A^TA) = \lambda_k(A^TX^TXA + \Delta_k) \geq \lambda_k(A^TX^TXA), \end{align*} which implies \begin{align*} \sigma_1(X)\sigma_k(A) \geq \sigma_k(XA). \end{align*} This shows \begin{align*} \sigma_k(A) \geq \sup_{\substack{X \in \real^{k \times n} \\ X \neq 0}}\frac{\sigma_k(XA)}{\sigma_1(X)}. \end{align*} The proof to the other inequality is analogous.