3
$\begingroup$

Let $$B:= \begin{bmatrix} O_m & A \\ A^\ast & O_n \end{bmatrix}$$ where $A$ is an $m \times n$ matrix. Find the relationship between the two:

  • Singular values and singular vectors of $A$.

  • Eigenvalues and eigenvectors of $B$.


Try

Let $|\lambda_1| \ge \cdots \ge |\lambda_{m+n}|$ and $x_1, \cdots, x_{m+n}$ are eigenvalues and corresponding eigenvectors of $B$. (Note that eigenvalues are real, since $B$ is Hermitian.

Let $\eta_1 \ge \cdots \ge \eta_{k} \ge 0$ and $u_1, \cdots, u_m$ and $v_1, \cdots, v_n$ be singular values and corresponding left- and right- singular vectors of $A$, with $k := \min\{m,n\}$.

Note that $B^2 = B^\ast B= \begin{bmatrix} A A^\ast & 0 \\ 0 & A^\ast A \end{bmatrix}$ thus $B^2 x_i = B \lambda_i x_i = \lambda_i^2 x_i, \forall i$. Therefore,

$$ B^2x_i = \begin{bmatrix} AA^\ast x_i^{(1)} \\ A^\ast A x_i^{(2)} \end{bmatrix} = \begin{bmatrix} \lambda_i^2 x_i^{(1)} \\ \lambda_i^2 x_i^{(2)} \end{bmatrix} $$

where $x_i := [\left(x_i^{(1)}\right)^T_m | \left(x_i^{(2)} \right)^T_n]^T_{m+n}$.


I have noticed that $x_i^{(2)}$, $\lambda_i^2$ are eigenpairs of $A^\ast A$, thus by the definition of singular value, $|\lambda_i|$ are the singular values of $A$.

By the way, from SVD of $A = U \Sigma V^\ast$, we have

$$ A^\ast A V = V\Sigma^2_{n \times n} \Leftrightarrow A^\ast A v_j = \eta_j^2 v_j \forall j=1,\cdots, k \\[7pt] A A^\ast U = U\Sigma^2_{m \times m} \Leftrightarrow A A^\ast u_l = \eta_l^2 u_l \forall l=1,\cdots, k $$


Question

Since $B$ has $(m+n)$ different(possibly same) $\lambda_i$ (i.e. $i = 1, \cdots, m+n)$, from

$$ B^2x_i = \begin{bmatrix} AA^\ast x_i^{(1)} \\ A^\ast A x_i^{(2)} \end{bmatrix} = \begin{bmatrix} \lambda_i^2 x_i^{(1)} \\ \lambda_i^2 x_i^{(2)} \end{bmatrix} $$

we have $(m+n)$ formulas for $x_i^{(1)}$,$x_i^{(2)}$, i.e.

$$ AA^\ast x_i^{(1)} = \lambda_i^2 x_i^{(1)} \\ A^\ast A x_i^{(2)} = \lambda_i^2 x_i^{(2)} \\ $$

for $i = 1,\cdots, m+n$, but for

$$ A^\ast A V = V(\Sigma^T\Sigma)_{n \times n} \Leftrightarrow A^\ast A v_j = \eta_j^2 v_j \forall j=1,\cdots, k \\[7pt] A A^\ast U = U(\Sigma\Sigma^T)_{m \times m} \Leftrightarrow A A^\ast u_l = \eta_l^2 u_l \forall l=1,\cdots, k $$

we only have $k$ formulas. So I'm stuck at specifying the relationship.

Is there anyone to help solving the problem?

$\endgroup$
4
$\begingroup$

The eigenvectors/eigenvalues of matrix $B$ can be easily constructed out of singular vectors/values of matrix $A$. Here is how.

In order to have simple explanations, let us consider the case where $A$ is square, i.e. with $m=n$ (for the rectangular case, see below).

Let

$$A=U\Sigma V^T=\sum_{k=1}^n \sigma_k U_kV_k^T$$

be the SVD of $A$ where $U_k$ and $V_k$ $(k=1 \cdots n)$ denote the columns of $U$ and $V$ resp.

Recall that : $$AV_k = \sigma_k U_k \ \ \text{and} \ \ A^TU_k = \sigma_k V_k\tag{1}$$

Therefore,

$$\underbrace{\begin{pmatrix} O & A \\ A^T & O \end{pmatrix}}_B \underbrace{\begin{pmatrix} U_k \\ V_k \end{pmatrix}}_{X_k}=\sigma_k\underbrace{\begin{pmatrix} U_k \\ V_k \end{pmatrix}}_{X_k} \ \text{and} \ \underbrace{\begin{pmatrix} O & A \\ A^T & O \end{pmatrix}}_B \underbrace{\begin{pmatrix} \ \ \ U_k \\ -V_k \end{pmatrix}}_{Y_k}=-\sigma_k\underbrace{\begin{pmatrix} \ \ \ U_k \\ -V_k \end{pmatrix}}_{Y_k}\tag{2}$$

which means that each singular triple $(\sigma_k,U_k,V_k)$, generates two eigenpairs : $(\sigma_k, X_k)$ and $(-\sigma_k, Y_k)$.

From this remark, it is possible to retrieve in particular an eigendecomposition of matrix $B$.


If $A$ is $n \times p$ with, say, $n<p$, the only difference is that, to these eigenvalues $\pm \sigma_k$, one must add eigenvalue $0$ with an order of multiplicity $p-n$ ; up to you for exhibiting the corresponding eigenvectors (don't forget that the last $p-n$ elements of $V$ constitute a basis of $\ker A$).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.