Questions tagged [singular-values]
This tag is for questions relating to 'Singular Value'. The term “singular value” relates to the distance between a matrix and the set of singular matrices
44
questions
64
votes
4
answers
34k
views
How unique are $U$ and $V$ in the singular value decomposition $A=UDV^\dagger$?
According to Wikipedia:
A common convention is to list the singular values in descending order. In this case, the diagonal matrix $\Sigma$ is uniquely determined by $M$ (though the matrices $U$ and ...
- 1,275
61
votes
2
answers
73k
views
Why does the spectral norm equal the largest singular value?
This may be a trivial question yet I was unable to find an answer:
$$\left \| A \right \| _2=\sqrt{\lambda_{\text{max}}(A^{^*}A)}=\sigma_{\text{max}}(A)$$
where the spectral norm $\left \| A \right \...
- 820
187
votes
7
answers
217k
views
What is the difference between "singular value" and "eigenvalue"?
I am trying to prove some statements about singular value decomposition, but I am not sure what the difference between singular value and eigenvalue is.
Is "singular value" just another name for ...
- 1,879
9
votes
3
answers
1k
views
Gradient of $A \mapsto \sigma_i (A)$
Let $ A $ be an $m \times n$ matrix of rank $ k \le \min(m,n) $. Then we decompose $ A = USV^T $, where:
$U$ is $m \times k$ is a semi-orthogonal matrix.
$S$ is $k \times k$ diagonal matrix , of ...
- 2,136
5
votes
2
answers
5k
views
How to show the von Neumann trace inequality?
Let $A,B$ have the appropriate size. How can we show the von Neumann trace inequality?
$$ \mbox{Tr}(AB) \leq \sum_{i=1}^n \sigma_{A,i}\sigma_{B,i} $$
Also, what is the intuition behind this inequality?...
user494522
40
votes
6
answers
33k
views
How can you explain the Singular Value Decomposition to non-specialists?
In two days, I am giving a presentation about a search engine I have been making the past summer. My research involved the use of singular value decompositions, i.e., $A = U \Sigma V^T$. I took a high ...
- 3,452
15
votes
1
answer
4k
views
Singular values of $AB$ and $BA$ where $A$ and $B$ are rectangular matrices
This question is a generalisation of Eigenvalues of $AB$ and $BA$ where $A$ and $B$ are rectangular matrices which itself is a generalisation of Eigenvalues of $AB$ and $BA$ where $A$ and $B$ are ...
- 213
5
votes
2
answers
3k
views
Interlacing Theorem on Singular Values
Does the Cauchy's interlacing theorem hold for "singular values" of matrices too? I saw on this publication first Theorem that it does. It states that singular values of a matrix interlace the ...
- 294
16
votes
1
answer
8k
views
Sum of eigenvalues and singular values
How one can prove that for a matrix $A\in \mathbb{C}^{n\times n}$ with eigenvalues $\lambda_i$ and singular values $\sigma_i$, $i=1,\ldots,n$, the following inequality holds:
$$ \sum_{i=1}^n \sigma_i(...
- 711
3
votes
1
answer
2k
views
Gradient of the spectral norm of a matrix
Let $X \in \mathbb{R}^{a \times b}$ and
$$\|X\|_2 = \sigma_{\max}(X) = \sqrt{\lambda_{\max} \left( X^T X \right)}$$
How can I compute $\nabla_X \|AX\|_2$, where $A \in \mathbb{R}^{c \times a}$ is ...
- 665
2
votes
1
answer
1k
views
Show adding rows to a non-singular square matrix will keep or increase its minimum singular value
I realize the following problem can be summarized as to show
Adding rows to an $n \times n$ non-singular matrix will keep or increase its minimum singular values.
Let $\bf A$ be an $m \times n$ ...
- 5,576
21
votes
1
answer
29k
views
Are the singular values of the transpose equal to those of the original matrix?
It is well known that eigenvalues for real symmetric matrices are the same for matrices $A$ and its transpose $A^\dagger$.
This made me wonder:
Can I say the same about the singular values of a ...
- 3,496
10
votes
1
answer
14k
views
Condition number of a rectangular matrix
From what I understand, the condition number of a rectangular matrix $A$ is its largest singular value divided by its smallest nonzero singular value
$$\kappa(A) := \frac{\sigma_1 (A)}{\sigma_n (A)}$...
- 269
6
votes
1
answer
6k
views
Proof or reference to the Weyl inequalities?
Does anyone know a proof or reference to the following result?
Suppose that $A, B$ are both $m \times n$ real matrices. Then for all $1 \leq k \leq \min\{m, n\}$,
$$|\sigma_k(A) - \sigma_k(B)| \...
- 3,690
5
votes
1
answer
495
views
Can we choose smoothly the singular vectors of a matrix?
Let $X$ be the space of all real $n \times n$ matrices, with strictly negative determinant, and pairwise distinct singular values. $X$ is an open subset of the space of all real square matrices. Is ...
- 25.1k
4
votes
3
answers
2k
views
Can a nonsingular square matrix be made singular by changing exactly one element or vice versa?
Given a nonsingular square matrix $A$, can changing just one element make it singular?
Given a singular square matrix $A$, can changing just one element make it nonsingular?
For $1$) I ...
- 5,857
4
votes
1
answer
7k
views
Frobenius and operator norms of rank 1 matrices
$\newcommand{\opnorm}[1]{\left\| #1 \right\|_{\mathrm{op}}}
\newcommand{\norm}[1]{\left\| #1 \right\|}$
Suppose we have $X = x_1 x_2^\top \in \mathbb{R}^{n \times d}$ a rank-1 matrix which is non-...
- 3,840
3
votes
2
answers
84
views
Nontrivial lower-bound for $\inf_{x \in \Delta_n} \|Gx\|$
Let $G$ be and $m \times n$ matrix of full-rank $n \le m$ in particular, and let $\Delta_n := \{x \in \mathbb R^n \mid x_1,\ldots,x_n \ge 0,\;\sum_{i=1}^n x_i = 1\}$ be the $(n-1)$-dimensional unit ...
- 9,545
1
vote
1
answer
169
views
Proving that left and right eigenvector are equal using singular values
Let $A$ be a square (complex) non-Hermitian matrix. Let us note by $s_i$ its singular values (in decreasing order) and by $a_i$ its eigenvalues (also in decreasing order, wrt. their absolute value).
...
0
votes
1
answer
69
views
Singular value relation for an LMI
Can we have the following?
For matrices $A$ and $B$, if $A \succeq B \implies \overline\sigma(A) \ge \overline\sigma(B)$?
where $\overline\sigma(\cdot)$ means the largest singular value.
- 1
8
votes
2
answers
458
views
Evaluating $K\big(\frac{3-\sqrt{7}}{4\sqrt{2}}\big)$
On MSE, I have seen derivations of the elliptic integral special values
$$K(1/\sqrt{2})=\frac{\Gamma^2(1/4)}{4\sqrt{\pi}}$$
$$K(\tan(\pi/8))=\frac{\sqrt{\sqrt{2} +1} \Gamma (1/8)\Gamma (3/8)}{2^{13/4}\...
- 39.8k
5
votes
1
answer
257
views
Minimize $x^*(A+A^*)x$ such that $x^*A^*Ax=1$ and $x^*x=1$
Given $A\in\mathbb{C}^{n\times n}$, such that it has singular values larger than $1$ and smaller than $1$,
\begin{array}{ll} \underset{x\in\mathbb{C^n}}{\text{minimize}} & x^*(A+A^*)x.\\ \text{...
- 1,910
4
votes
1
answer
1k
views
Does a matrix with negative eigenvalues have singular values?
What if the eigenvalues of a matrix $A$ are all negative? Does that simply mean there is no singular value for this particular matrix?
I can't calculate the conditional number or matrix $2$-norm for ...
- 51
4
votes
3
answers
1k
views
Special name for matrices with the same singular values?
Suppose there exist matrices $A \in \Bbb C^{m \times n}$ and $B = U_1 A U_2$, where $U_1 \in \Bbb C^{m \times m}$ and $U_2 \in \Bbb C^{n \times n}$ are unitary matrices (not necessarily related to ...
- 521
4
votes
1
answer
231
views
A Quotient Representation of Singular Values of Symmetric Matrix
The question goes as:
Let $\sigma_1(A) \geq \sigma_2(A) \geq \cdots \geq \sigma_r(A)$ be all (non-zero) singular values of an order $n$ real matrix $A$. Prove:
\begin{align*}
\sigma_k(A) = \sup_{\...
- 14.1k
4
votes
1
answer
6k
views
Why can't singular values be complex numbers?
Reading both this paper (p. 4) and the Wikipedia article regarding singular value decomposition, they state that the diagonal matrix ${\boldsymbol {\Sigma }}$ in $$\mathbf {M} =\mathbf {U} {\...
user168764
3
votes
0
answers
662
views
Continuity of Singular Value Decomposition
I am trying to show that the singular value decomposition is continuous.
For a matrix $A$ of size $m\times n$, let $s = \min\{m,n\} =n$, i.e, $(m \ge n)$.
Let $A = U_A\Sigma_A V_A^T$ be a SVD of $A$ ...
- 1,944
3
votes
3
answers
695
views
Singular value decomposition works only for certain orthonormal eigenvectors, not all?
I'm trying to find the SVD of the following matrix:
$$A=
\begin{pmatrix}
1 & 1 \\
2 & -2 \\
2 & 2 \\
\end{pmatrix}
$$
I found the eigenvalues and vectors for $A'A$:
$$
\begin{array}{...
- 5,243
2
votes
1
answer
4k
views
Clarification on the SVD of a complex matrix
Given a matrix $G$, let its singular value decomposition be
$$G = Y \Sigma U^H$$
In the Hermitian matrix $G^H = U \Sigma Y^H$, why doesn't $\Sigma ^H$ appear instead of $\Sigma$?
If $G$ is a ...
- 1,245
2
votes
1
answer
272
views
Closest conformal matrix to a given matrix
$\newcommand{\SOn}{\operatorname{SO}_n}$
$\newcommand{\COn}{\operatorname{CO}_n}$
$\newcommand{\Sym}{\operatorname{Sym}_n}$
$\newcommand{\Skew}{\operatorname{Skew}_n}$
$\newcommand{\dist}{\...
- 25.1k
2
votes
1
answer
167
views
If the matrix $A$ is perturbed by a symmetric matrix $E$, then its eigenvalues do not move by more the $||E||$
"Matrix Computations" by Golub and Van Loan, 4th ed, Chapter 2, Corollary 2.4.4 says:
If $A\in R^{m\times n} and E\in R^{m\times n}$, then
$$\sigma_{min} (A+E)\geq \sigma_{min}(A)-||E||_2$$
$\sigma$...
- 79
2
votes
2
answers
321
views
How to calculate explicitly some matrix norm?
I want to calculate the norm of the matrix
$$A = \left(\begin{array}{cc} 1&1 \\ 0&1\end{array}\right).$$
The norm is
$$\Vert A \Vert_2 = \sup_{\Vert v \Vert = 1}\Vert Av \Vert.$$
I can show ...
- 492
2
votes
1
answer
419
views
Relationship between matrices whose singular values are the same
Motivation: I have two different matrices in $\mathbb{R}^{1000 \times 2048}$. $A_1$ is coming from an sparse optimization process whose objective is creating as much as zeros in $A_1$. In this sense, ...
- 175
2
votes
2
answers
472
views
Maximization of $\infty$-norm with $2$-norm constraint
Suppose we have a matrix $\mathbf B \in \mathbb R^{n\times n}$. Is there an analytical solution to the following problem?
\begin{equation}
\begin{aligned}
&\max_{\mathbf x \in \mathbb R^n} &&...
- 1,541
2
votes
2
answers
468
views
Is the set of matrices with distinct singular values connected and dense?
$\newcommand{\GLp}{\operatorname{GL}_n^+}$
$\newcommand{\SO}{\operatorname{SO}_n}$
Consider the set
$$ X=\{ A \in \GLp \,\,|\,\, \text{ all the singular values of } A \text{ are distinct } \},$$
where ...
- 25.1k
1
vote
1
answer
94
views
How to find a matrix $X$ such that $Q \operatorname{vec}(X)=0$ and enforce $det(X) \neq 0$
Assume (homogeneous) system of linear equations in $X$ of the form $Q\operatorname{vec}(X)=0$, where $Q$ is in general a tall non-square matrix. I want to find a non-trivial matrix $X$ (i.e. $X \neq 0$...
- 878
1
vote
1
answer
716
views
Do singular values change when an arbitrary matrix is multiplied by a unitary matrix?
Let X be an arbitrary matrix, and let U be a unitary matrix.
Is it so that the singular values of X are the same as the singular values of UX?
Does the same apply for XU?
- 31
1
vote
2
answers
1k
views
The smallest non-zero singular value of AB
For a matrix $A$, let $σ_{min}(A)$ denote the smallest $\textbf{non-zero}$ singular value of A.
I saw some materials (e.g. https://pdfs.semanticscholar.org/e74f/89ac85239811b295787619e73c99fc867724....
- 174
1
vote
0
answers
63
views
Can I invoke the continuity of eigenvalues to prove this result?
Suppose I have two sequences $(A_n)$, $(B_n)$ of $n\times n$ symmetric real matrices (growing sizes) such that $||A_n-B_n||_F\to 0$ as $n \to \infty$, where $||\cdot||_F$ denotes the Frobenius norm. I ...
- 4,740
1
vote
1
answer
964
views
If the absolute value of every eigenvalue of a matrix is smaller than 1, is the maximum singular value smaller than 1?
I am just curious. If the absolute value of every eigenvalue of a matrix is smaller than 1, is the maximum singular value smaller than 1? This is related to my previous question Any relation between ...
- 5,576
1
vote
1
answer
2k
views
Singular Values of Symmetric Matrix
I saw the following claim in this thread:
How to compute the SVD of a symmetric matrix?
Claim: The singular values of a symmetric matrix $A$ are the absolute values of its eigenvalues.
I ...
- 451
1
vote
1
answer
1k
views
Summation of singular values
If $A \in \mathbb{R}^{m\times n}$, then show that $$\sum_{s=1}^{r}\sigma_s(A)=\text{max}\{\text{trace}(U^TAV): U \in \mathbb{R}^{m\times r}, V\in \mathbb{R}^{n\times r}, \text{and} \ U^TU=V^TV=I_r\}.$$...
- 991
0
votes
1
answer
59
views
On invertibility of $A+E$ where $||E||_2<$ smallest singular value of $A$ and $||A^{-1}E||_2<1$
Let $A,E \in M_n(\mathbb C)$ . Suppose $\sigma_\min >0$ be the smallest singular value of $A$ and $||E||_2 < \sigma_\min$. Suppose $||A^{-1}E||_2 <1$. Then how to show that $A+E$ is ...
- 3,705
0
votes
1
answer
255
views
Singular value inequality for block matrices
Suppose that $A \in \mathbb{C}^{m \times n}$, $m \geq n$, has the block form
$$A =
\begin{bmatrix}
A_1 \\
A_2
\end{bmatrix}
$$
where $A_1 \in \mathbb{C}^{n \times n}$ and
$A_2 \in \mathbb{C}^{(m-n) \...
- 409