Questions tagged [singular-values]
This tag is for questions relating to 'Singular Value'. The term “singular value” relates to the distance between a matrix and the set of singular matrices
460
questions
0
votes
0
answers
45
views
Why the singular values of $A$ is the positive square root of the eigenvalues of $A^{\top}A$?
I've tried to look for the answer to this curiosity over the internet, but I couldn't find any explanation that is clear enough.
So here's the question: consider a linear transformation $L\colon\...
- 47
0
votes
0
answers
39
views
Equality singular value $\sigma_1(A)=\sup_{\lVert x\rVert=1}\lVert Ax\rVert=\sup_{\lVert x\rVert =1,\lVert y\rVert=1}\langle Ax,y\rangle$
I am trying to solve:
$\sigma_1(A) = \sup\limits_{\lVert x \rVert = 1} \ \lVert A x \rVert = \sup\limits_{\lVert x \rVert = 1, \lVert y \rVert = 1} \langle Ax,y\rangle$
where $\sigma_1(A)$ is the ...
- 23
0
votes
0
answers
18
views
Scale Invariant Singular Value Decomposition
I am looking for a reference to the concept of Scale Invariant SVD which is mentioned in the "variations and generalisations" section of the Wikipedia article for SVD:
https://en.wikipedia....
- 31
1
vote
0
answers
20
views
Lower bound on distance between matrix values allows for lower bound on smallest singular value
Say I have a $2 \times 2$ matrix $M$ with the property that $$||M_{ij} - M_{i'j'}|| \geq k$$ for some constant $k$ and all $i, j, i', j' \in \{1, 2\}$. This would imply that the entries of $M$ are ...
- 11
0
votes
0
answers
26
views
For $m \times n$ matrix $A$ with $m \ge n$, show ${\lVert A^+ \rVert}_2 = 1/\sigma_n$ where $A^+ = (A^*A)^{-1} A^*$ [duplicate]
For $m \times n$ matrix $A$ with $m \ge n$, show that the norm of the pseudoinverse ${\lVert A^+ \rVert}_2 = 1/\sigma_n$ where $A^+ = (A^*A)^{-1} A^*$ and $\sigma_n$ is the nth singular value of $A$.
...
- 2,639
0
votes
1
answer
77
views
Understanding the proof of the sum of eigenvalues and singular values
I am trying to Understand this proof . So far I understand everything but the part where the author says the following:
"Due to Schur decomposition, there exist a unitary matrix $U$ and an upper ...
0
votes
1
answer
47
views
The maximal singular value of a block matrix.
(1) About a week ago I have asked the question and got a beautiful explanation. After that I started to consider the relationship of the singular values of the big matrix $A$ and the small matrix $A_{...
0
votes
1
answer
42
views
SVD of product of diagonal and unitary matrices
Given two (possibly rectangular) diagonal matrices $\Sigma_\text{L}$ and $\Sigma_\text{R}$ with nonnegative elements, what can we say about the singular value decomposition of
$$\Sigma_\text{L} X \...
- 63
0
votes
0
answers
61
views
how to understand singular values geometrically like eigenvalues?
After a linear transformation, some vectors may not change direction, they only scale by a number. The scaling factor of those vectors is called eigenvalue.
Can we think of singular values in this ...
- 1
1
vote
0
answers
21
views
SVD distribution of linearly transformed Gaussian ensemble
The joint pdf of the singular values of the $m \times n$ Gaussian ensemble $X = x_{i,j} $, where the $x_{i,j} $'s are independent Normal(0,1) samples and the eigenvalues of the associated Wishart ...
- 11
0
votes
0
answers
17
views
Smallest Singular Value of submatrices from a column-orthogonal matrix
Suppose we have a column-orthogonal matrix $\mathbf {U}\in\mathbb{R}^{n\times p}$, satisfying $\mathbf {U}^{\top}\mathbf {U}=\mathbf {I}_p$. We select $m<n$ rows of $\mathbf {U}$ randomly and get $\...
- 13
0
votes
0
answers
64
views
Uniqueness of singular vectors (Theorem 4.1 Trefethen & Bau)
I am looking for some clarifications in the uniqueness portion of the proof of Theorem 4.1 of Trefethen & Bau's Numerical Linear Algebra.
The definition of the SVD and the proof exerpt from the ...
- 177
1
vote
1
answer
52
views
Bounds on the eigenvalues of $\bf W D W^\top$
Given a diagonal matrix $\bf D$ with diagonal entries $d_{ii} \in [0,1]$ and a matrix $\bf W$ with singular values $\sigma_i ({\bf W}) \in [0,1]$, can it be proven that the eigenvalues of $\bf W D W^\...
2
votes
1
answer
129
views
Singular values of uniform random points on hypersphere?
This question is motivated by a self-supervised learning problem in machine learning, but I'll try to strip out as many unnecessary details as possible. In this setting, we have large datasets and we ...
- 452
2
votes
1
answer
72
views
Singular value of identity minus 1-rank matrix
Given vectors $u,v\in \mathbb{R}^d$, I wonder what we can say about the minimum singular value of $I-uv^\top$? I know that when $u=v$, this matrix is symmetric so it is not hard to compute this. ...
- 115
1
vote
1
answer
29
views
Least nonzero singular value of $A^{-1} P$ for some invertible matrix $A$ corresponds to the reciprocal of the largest singular value of $A \circ P$?
Let $A$ be $n \times n$ real invertible matrix, and $P : \mathbb{R}^n \to \mathbb{R}^n$ be some orthogonal projection.
Then, what would be the relation between the smallest nonzero singular value of $...
- 7,088
0
votes
0
answers
21
views
Why are the singular values equal to the first partial derivatives.
I am studying computer science so please go easy on me.
I am also too bad at math to extract the mathematical essence that is needed to answer this question so I'm just gonna explain the whole setup.
...
0
votes
0
answers
14
views
(need explanation) Compute singular values of $ X^{(t)} $ via reduced gramians
background information: I just started with the topic of singular values. Please be understanding.
groundwork:
So lets say we a matrix $ X^{(t)} $.
$U_t$ is a basis for the columb spaces of $ X^{(t)} $...
- 55
0
votes
0
answers
80
views
Decide on near zero eigenvalues
I am using SVD to solve a homogeneous system of $N$ linear equations in $12$ variables, where $N \gg 12$, in the least-squares sense. In order to determine the null-space of the $N \times 12$ matrix, ...
- 31
1
vote
0
answers
80
views
SVD-based pseudoinverse solution sensitivity to equation linear combinations
My problem is of theoretical nature. Given an overdetermined system of $m$ equations in $n$ unknowns, $\bf A x = b$, where $m \gg n$ and
$$ {\bf A} = \begin{bmatrix} — {\bf a}_1 — \\ — {\bf a}_1 — \\ \...
- 11
1
vote
0
answers
40
views
When does a singular value of a square invertible matrix become an eigenvalue?
Let $A$ be an $n \times n$ invertible matrix with real entries. I know that its largest singular value, $\sigma_{\max}$ is equal to its operator norm. Moreover, there are singular vectors $u, v \in \...
- 7,088
0
votes
1
answer
72
views
Bounds on singular values of a Hankel-like matrix
Suppose I have a matrix where each successive row is a left-shift of the previous row (with a new value coming in on the right), e.g.,
$$A = \begin{bmatrix} 1 & 2\\ 2&3\\ 3&4 \end{bmatrix}...
- 11
1
vote
0
answers
30
views
Is the $k$-th elementary symmetric polynomial of the singular values the nuclear norm of the $k$-th alternating power of the matrix?
If $A$ is a complex $m$ by $n$ matrix, thus representing a $\mathbb{C}$-linear map from $\mathbb{C}^n$ to $\mathbb{C}^m$, we denote by $s_1, \ldots , s_r$ its singular values. Let $e_k$ denote the $k$-...
- 5,186
2
votes
0
answers
65
views
Maximum singular value via nonconvex QCQP
Finding the extremal singular triplet $(σ, u, v)$ of a generic real $m×n$ matrix $A$ can be formalized as a nonconvex quadratically-constrained quadratic program (QCQP):
$$σ = \max_{u,v}\quad u^⊤ A v ...
- 11.4k
1
vote
0
answers
62
views
Singular vectors of sums of outer products
I have a symmetric PSD matrix
$$ P = \sum_{i = 1}^N p_ip_i^\top \in \mathbb{R}^{n \times n} $$
where $p_i \in \mathbb{R}^n\ \forall i$. Its SVD is $P = U\Sigma_pU^\top$. I also have another sum
$$A = \...
- 33
1
vote
0
answers
49
views
Inequalities between biggest eigenvalue and singular values for a traceless matrix [closed]
Let $A \in {\Bbb C}^{4 \times 4}$ be a traceless matrix with Frobenius norm smaller than $1$. Let $\lambda_i$ be $i$-th largest (by modulus) eigenvalue of matrix $A$. Let $\sigma_i$ be $i$-th largest ...
0
votes
0
answers
25
views
Lower bound on singular values of a bordered matrix
Let
$M = J + H =
\begin{bmatrix}
0 & \vec{1}^{\intercal}\\
\vec{1} & K + \gamma^{-1}I
\end{bmatrix} =
\begin{bmatrix}
0 & \vec{1}^{\intercal}\\
\vec{1} & 0
\end{bmatrix}_J +
\begin{...
- 33
0
votes
0
answers
31
views
Given Gaussian "noise" matrix $G$ and matrix products $AG$, and $A^\intercal AG$, solve for $A$
Given Gaussian "noise" matrix $G$ and matrix products $AG$, and $A^\intercal AG$, solve for $A$
Let $A \in \mathbb{R}^{m \times n}$ be a matrix with rank $k$. Consider the following ...
- 2,639
0
votes
0
answers
25
views
Why the number of non-zero singular values is $n-1$ for $X\in \mathbb{R}^{n \times m}$ when $n\ll m$
There's a statement in the paper Phatak (1997)
In most spectroscopic problems, the number of non-zero singular values is $n-1$ for $X\in \mathbb{R}^{n \times m}$ when $n\ll m$. Consequently, at most $...
- 320
1
vote
1
answer
79
views
Upper bound for sum of singular values of symmetric hollow matrix
Let $\bf{H}$ be $n\times n$ real symmetric matrix with zero diagonal and entries $h_{ij} \in [0, M]$. Denoting its eigenvalues $\lambda_1 \leq \dots \leq \lambda_n$, I want to improve the upper bound $...
- 43
2
votes
1
answer
84
views
Singular values of matrix. Estimate norm of the expression
Let singular values of matrix A are written in vector $s = (s_1, \ldots, s_n)$ in decreasing order ($n \ge 2$). Let $x,y$ are orthogonal vectors with $\|x\| = 1$ and $\|y\| = 1$.
What is the biggest ...
- 101
0
votes
1
answer
40
views
Largest Singular Value of the Exponential of a Normal Matrix
I am trying to work out a relation between the largest singular value $\sigma_1(Q)$ of a normal matrix $Q$, and the largest singular value of $e^{Q}$. In an ideal world for the wider context of my ...
- 11
0
votes
0
answers
70
views
Connection between ratio of magnitude of eigenvalues and ratio for singular values
Let $A$, $B$ be a square, complex $4 \times 4$ matrices such that $Tr(A) = Tr(B) = 0$ Let $s_i(X)$ be i-th biggest singular value of matrix X. Let $\lambda_i(X)$ be i-th biggest eigenvalue (by ...
0
votes
0
answers
27
views
Estimate the trace of the inverse matrix via matvec
Here we are given some square matrix $A \in R^{n \times n}$ (you can assume $A$ to be symmetry or p.s.d if needed). By given I mean a black box implementation of the matrix-vector multiplication $x \...
- 21
1
vote
0
answers
40
views
Can we express the spectral norm of a matrix $F$ in terms of the singular values of the sub-matrix $K$?
Let $\bf K$ be a generic $n \times n$ matrix, and let
$$ {\bf F} := \begin{bmatrix} 0 & {\bf 1}_n^\top \\ {\bf 1}_n & {\bf K} \end{bmatrix} $$
Can we express the spectral norm of $\bf F$ in ...
- 33
-1
votes
1
answer
48
views
Eigenvalues Bounded by Singular Values Proof
I'm trying to prove the inequality for a real square matrix $M$
$\sigma_\min(M)\leq|\lambda_i|\leq\sigma_\max(M)$ for all $i$
We have that:
$\lambda_\max(M)=\sup_{x\neq0}\frac{x^TMx}{||x||_2^2}$
$\...
- 13
1
vote
1
answer
44
views
Bound quotient of maximal and minimal singular values
I am working on a problem where the following quantity emerges:
$$
\frac{\sigma_{\text{max}}(J-M)}{\sigma_{\text{min}}(J+M)}
$$
where $J$ is the canonical symplectic matrix, $M^T=M$ and $M$ is ...
- 754
1
vote
1
answer
277
views
Why the singular values in SVD are always hierarchical/descending?
Please, I'm trying to understand why singular values (SV) are always hierarchical/descending. At the beginning of my studies, I thought that the hierarchy of sigmas ($ \sigma_1 \geq \sigma_2 \geq ... \...
- 19
0
votes
1
answer
86
views
Non-Symmetric Real Matrix with eigen values $\in $ [0,1] but largest singular value > 1
Matrix , M has all eigenvalues $\in$ [0,1], but on simulation, I can see that the largest singular value is >1.
a) Can anyone give an example of such a matrix in toy cases like 2x2
b) Can some ...
- 113
1
vote
0
answers
49
views
Name & notation for the collection of a matrix's singular values
I've often seen the notion of the spectrum of a matrix $A \in \mathbb{F}^{n \times n}$ (Wikipedia article),
$$\sigma(A) := \{ \lambda \in \mathbb{F} \mid \lambda \text{ is an eigenvalue of $A$} \}$$
(...
- 42.1k
2
votes
1
answer
191
views
Is the condition number (SVD) a measure of the orthogonality of a matrix?
We say a matrix is well-conditioned when its condition number is close to 1, and is ill-conditioned when the condition number is large. I'm puzzled whether the condition number has a simple ...
- 3,055
1
vote
0
answers
104
views
Intuitive reason as to why the spectral norm is the largest singular value
Why is the spectral norm, the largest singular value? I understand the proof behind this, but I can not intuitively explain why the spectral norm is the largest singular value. It makes sense that it ...
- 11
0
votes
0
answers
30
views
Relationship between singular values, traces and Hermitian conjugate
In my free time, I am working on a following problem inspired by this paper - arxiv.org/abs/0711.2613
Let $A$, $B$ be zero-trace $4 \times 4$ matrices that meet following condition $\textit{Tr}(A^{\...
0
votes
1
answer
140
views
How can I prove this theorem for the maximum singular value of a matrix $A$?
Let $A \in \mathbb C^{p \times q}$ and $B \in \mathbb C^{q \times p}$. Let $\bar\sigma(A)$ be the largest singular value of $A$, and let $\rho(AB)$ be the spectral radius of $AB$, which is the maximum ...
- 1,225
0
votes
1
answer
53
views
Are the following statements true regarding the singular values of a real matrix?
Let $A\in \mathbb{R}^{m\times k}\ (k<m)$ be a real matrix. Suppose, $\forall x \in \mathbb{R}^{k}$, and for a $\delta \in (0,1)$, the following inequality holds:
\begin{equation}
(1-\delta) \|x\|...
- 389
1
vote
1
answer
51
views
Given a symmetric matrix $H$, can we prove $x^THx\le \sigma(H)\|x\|_2^2$?
Given a symmetric matrix $H$, can we prove that $$x^THx\le \sigma(H)\|x\|_2^2$$ where $ \sigma(H)$ is the maximal singular value of $H$?
- 2,783
1
vote
1
answer
51
views
How to calculate the eigenvalues of $A^\ast A$ where $A$ is a companion matrix?
Let $$A=\left[\begin{array}{cc}0&-a_0\\I_{n-1}&\xi\end{array}\right]$$ be a companion matrix where $\xi=[-a_1\ \cdots\ -a_{n-1}]^T\in \mathbb{C}_{n-1}$. Hence, $$A^\ast A=\left[\begin{array}{...
- 197
0
votes
0
answers
77
views
Compute the singular values of convolution matrix through fast Fourier transform
Given any vector $\boldsymbol{x}=(x_1,x_2,\ldots,x_n)^\top\in\mathbb{R}^{n}$ with a certain kernel size $\tau\in\mathbb{N}^+$ ($1<\tau<n$), then its convolution matrix is given by
\begin{...
- 107
0
votes
1
answer
38
views
maximization of norm $||AXAb||$
Let $X\in \mathbb{R}^{m,n}$ and $A\in \mathbb{R}^{n,m}$, $b\in \mathbb{R}^{m}$.
How do we solve the following maximization problem?
$$\text{maximize}_{||X||=1} ||AXAb||_2$$
My thought was that we can ...
- 170
1
vote
1
answer
439
views
SVD of constituent blocks of a block diagonal matrix
Given a block diagonal matrix, is it possible to read out the SVD of its constituent blocks using the SVD of the block diagonal?
When I say block diagonal, it might not necessarily mean that the ...
- 3,333