Questions tagged [singular-values]
This tag is for questions relating to 'Singular Value'. The term “singular value” relates to the distance between a matrix and the set of singular matrices
478
questions
0
votes
0
answers
23
views
Generation of the notion of singular value in Banach space
Recall that the singular values of a $d \times d$ real or complex matrix $A$ are the non-negative square roots of the eigenvalues of the positive semidefinite matrix $A^{*} A$, and are denoted $\...
- 1
3
votes
1
answer
77
views
Linear Algebra Done Right, 4th Edition, problem 7.F.22
I'm trying to solve the following problem (from Axler's "Linear Algebra Done Right", 4th edition, problem 22 of section 7F):
Suppose $T \in \mathcal L(V,W)$. Let $n=\text{dim }V$ and let $...
- 300
0
votes
0
answers
22
views
Is it possible to align the complex phases of a diagonal matrix while preserving the given decomposition?
If $U$ is unitary, $A$ is complex symmetric, $t$ is the transpose, and $U_1^t A U_1$ yields a diagonal matrix $D_c$ with singular values of A with some complex phase, is there a way to make $U_2^t A ...
- 335
0
votes
1
answer
47
views
Minimal Singular Value Lower Bound: Diagonal Matrix + Skew-Symmetric Matrix + Its Sums of Rows
I have a matrix $M = D + S + A$ with $D, S, A \in \mathbb{R}^{n\times n}$. $D$ is a diagonal matrix for which $tr(D)$ is known. $S$ is a skew-symmetric matrix, while $A$ is diagonal as well and ...
- 1
1
vote
1
answer
148
views
Linear Algebra Done Right, 4th Edition, problem 7.F.13
I'm trying to solve the right inequality in the following problem (from Axler's "Linear Algebra Done Right", problem 13 of section 7F):
Suppose $S,T \in \mathcal L(V)$ are positive ...
- 300
4
votes
1
answer
121
views
Lower bound on the norm of product of 2 matrices
I've recently been reading Matrix Perturbation Theory by Stewart and Sun 1. I am confused about Theorem 3.9: for any unitarily invariant norm $\|\cdot\|$, one has $\|AB\|\geq \|A\|\inf_2(B)$. The ...
1
vote
0
answers
29
views
Exponentiating the singular values of a matrix without explicitly computing them
Let $X = U\Sigma V$ be a singular value decomposition of a real matrix $X$.
It is possible to "cube" the singular values of $X$ (that is to compute $X_3$ = $U\Sigma_3V$ where $\Sigma_3$ has ...
- 323
0
votes
0
answers
21
views
Can ${\rm cond}([A,B])$ be controlled by ${\rm cond}(A)$ and ${\rm cond}(B)$?
For a matrix $A \in \mathbb{R}^{n \times m}$, the condition number of $A$ is defined by ${\rm cond}(A) := \|A\|_2\|A^\dagger\|_2$, where $A^\dagger$ is the Moore-Penrose inverse of $A$.
Now consider ...
- 461
0
votes
0
answers
19
views
||A-B|| compared to the difference of their smallest singular values
I came cross a problem:
For two matrices $A \in \mathbb{R}^{m \times n}$, $B \in \mathbb{R}^{m \times n}$, if ${\rm rank}\ A = {\rm rank}\ B = r \leq \min \{m,n\}$, then
\begin{equation}
|\sigma_r(A)...
- 461
0
votes
0
answers
37
views
How to find best-fit 1-subspace in case when the first singular value has multiplicity greater than 1?
Span of the first k right singular vectors is the best-fit k-subspace.
But what if we have a singular value that has multiplicity d greater than one, then the corresponding singular vectors are not ...
2
votes
1
answer
39
views
Upper and lower bounding singular values of a nearly orthogonal matrix
Let $u_1, \dots, u_n$ be $n$-dimensional unit vectors and let $U = \begin{bmatrix}u_1 & \dots & u_n \end{bmatrix}$ be a matrix formed by stacking these vectors columnwise.
If $u_i^\top u_j = 0$...
- 300
0
votes
0
answers
29
views
Reasoning for reduced SVD factorization
I am aware that for any $m \times n$ matrix $A$, we can write:
Known 1: $A = U\Sigma V^T$ where $U$ is orthogonal and $m \times m$, $V$ is orthogonal and $n\times n$, and $\Sigma$ is diagonal and $m \...
- 601
2
votes
0
answers
31
views
Bound for singular values in terms of traces: a 6th power inequality
In the recent paper of Guth and Maynard on large values of Dirichlet polynomials https://arxiv.org/html/2405.20552v1 there is a linear algebra lemma 4.2. Bound for singular values in terms of traces:
...
- 747
0
votes
0
answers
29
views
Prove that $\sigma_n(\widehat{L}) \geq \sigma_n(L)/2$.
I recently come across a problem with respect to singular value as follow.
Suppose $\sigma_n(L) \geq 2\|L-\widehat{L}\|$. Then $\|\widehat{L}\| \leq 2\|L\|$ and $\sigma_n(\widehat{L}) \geq \sigma_n(L)/...
- 461
0
votes
0
answers
22
views
Largest singular value of anti-triangular matrix
I have an anti-triangular matrix that is not Hermitian (it is complex and symmetric). I would like to find a method to bound its largest singular value. Might that exist, in general?
Anti-triangular ...
- 443
1
vote
0
answers
23
views
Singular values as min max of absolute rayleigh quotient
Consider a real symmetric matrix M, satisfying $\mathbf{1}^\intercal M = \mathbf{1}^\intercal$ having eigenvalues $1=\lambda_1 \gt\|\lambda_2\| \geq \|\lambda_3\| .... \geq \| \lambda_n\|$, then can I ...
0
votes
0
answers
31
views
Singular value decomposition algorithm recommendations for smaller dense matrices
I am looking for recommendations for SVD algorithms for dense matrices. My supervisor specifically requested that I do not use external libraries for this (otherwise I'd likely use LAPACK and call it ...
0
votes
0
answers
46
views
what happens to singular values with one-rank update?
I have a square matrix $A$ and two vectors $a$ and $b$ such that $\sum a_i = 1$ and $b_j = 1$ for all $j$.
I would like to know if there is some way of expressing the relationship between the singular ...
- 43
0
votes
0
answers
7
views
Estimate the norm of pseudo inverse of the product of two matrices
Suppose $n \geq p \geq m$. Let $A \in \mathbb{R}^{m \times n}$ be of full row rank and $B \in \mathbb{R}^{n \times p}$ be of full column rank. If $A B$ is an $m$-by-$p$ matrix of full row rank. Can we ...
- 196
0
votes
0
answers
127
views
Singularity extraction: $\int_{-1}^{1}\int_{-1}^{1}\frac{dxdy}{\sqrt{(x-.3)^2+y^2}}$
$\int_{-1}^{1}\int_{-1}^{1}\frac{dxdy}{\sqrt{(x-.3)^2+y^2}}$ has the following physical meaning:
it is the potential of the uniform surface source distributed on a square $|x|,|y|<1$ observed at a ...
- 422
0
votes
0
answers
39
views
Solving Matrix Equation using SVD
I'm reading this paper by Bishop and Tipping.
They solve the equation
$$(SC^{-1} - I)W = 0$$
Where $W \in \mathbb{R}^{d \times q}$ and $S , C \in \mathbb{R}^{d \times d}$ and $C = WW^T + \sigma^2 I$ ...
- 43
0
votes
0
answers
20
views
Hankel Singular Values for diagonal state-space model
Let (A,B,C) be a diagonal and stable discrete-time LTI state space model:
\begin{align*}x(k+1)&=Ax(k)+Bu(k),\\ y&= Cx(k)\end{align*} with A being a diagonal matrix:
\begin{bmatrix}
\lambda_1 &...
0
votes
1
answer
94
views
How is effective rank (a ratio of nuclear norm and operator norm) defined in terms of eigenvalues?
I am reading this paper on implicit regularisation in gradient descent and I am having difficulty with the provided definition of effective rank. In the paper it is given as
$r(W) = \frac{||W||_*}{||W|...
- 93
4
votes
0
answers
81
views
Numerical Least squares estimation on the norm of the minimiser
I was looking at Proposition 3.2. in Applied Numerical Linear Algebra by JW Demmel, that states that when solving $$\min \|Ax - b\|_2 $$ if in the singular value decomposition $\sigma_{min}>0$ then ...
- 41
0
votes
0
answers
20
views
What are proper numerical methods to solve IVP at irregular singular point?
I have searched for numerical methods to solve IVP with irregular singular points but didn't find any.
What is a proper method I can use to solve such equations?
For example, we could consider the ...
- 1,266
0
votes
0
answers
7
views
$l_4$-norm of singular values of random matrix
Let $A$ be an $n \times d$ matrix ($n > d$) with iid standard Gaussian entries. Let $\lambda_1 \ge \lambda_2 \ge \dotsm \lambda_d > 0$ be the non-zero singular values of $A$.
What is known about ...
- 224
0
votes
1
answer
66
views
Deriving the SVD from the eigendecomposition
If $A$ is a rectangular matrix of dimensions $m\times n$, then $S_L=AA^T$ and $S_R=A^TA$ are square symmetric matrices. Hence, using the eigendecompostion we can write
$$
S_L=AA^T=U\Lambda_{S_L} U^T
$$...
- 3
0
votes
0
answers
56
views
Why the singular values of $A$ is the positive square root of the eigenvalues of $A^{\top}A$?
I've tried to look for the answer to this curiosity over the internet, but I couldn't find any explanation that is clear enough.
So here's the question: consider a linear transformation $L\colon\...
- 47
0
votes
0
answers
42
views
Equality singular value $\sigma_1(A)=\sup_{\lVert x\rVert=1}\lVert Ax\rVert=\sup_{\lVert x\rVert =1,\lVert y\rVert=1}\langle Ax,y\rangle$
I am trying to solve:
$\sigma_1(A) = \sup\limits_{\lVert x \rVert = 1} \ \lVert A x \rVert = \sup\limits_{\lVert x \rVert = 1, \lVert y \rVert = 1} \langle Ax,y\rangle$
where $\sigma_1(A)$ is the ...
- 23
1
vote
0
answers
115
views
Scale Invariant Singular Value Decomposition
I am looking for a reference to the concept of Scale Invariant SVD which is mentioned in the "variations and generalisations" section of the Wikipedia article for SVD:
https://en.wikipedia....
- 41
1
vote
0
answers
26
views
Lower bound on distance between matrix values allows for lower bound on smallest singular value
Say I have a $2 \times 2$ matrix $M$ with the property that $$||M_{ij} - M_{i'j'}|| \geq k$$ for some constant $k$ and all $i, j, i', j' \in \{1, 2\}$. This would imply that the entries of $M$ are ...
- 11
0
votes
0
answers
53
views
For $m \times n$ matrix $A$ with $m \ge n$, show ${\lVert A^+ \rVert}_2 = 1/\sigma_n$ where $A^+ = (A^*A)^{-1} A^*$ [duplicate]
For $m \times n$ matrix $A$ with $m \ge n$, show that the norm of the pseudoinverse ${\lVert A^+ \rVert}_2 = 1/\sigma_n$ where $A^+ = (A^*A)^{-1} A^*$ and $\sigma_n$ is the nth singular value of $A$.
...
- 2,803
0
votes
1
answer
97
views
Understanding the proof of the sum of eigenvalues and singular values
I am trying to Understand this proof . So far I understand everything but the part where the author says the following:
"Due to Schur decomposition, there exist a unitary matrix $U$ and an upper ...
0
votes
1
answer
154
views
The maximal singular value of a block matrix.
(1) About a week ago I have asked the question and got a beautiful explanation. After that I started to consider the relationship of the singular values of the big matrix $A$ and the small matrix $A_{...
- 57
0
votes
1
answer
86
views
SVD of product of diagonal and unitary matrices
Given two (possibly rectangular) diagonal matrices $\Sigma_\text{L}$ and $\Sigma_\text{R}$ with nonnegative elements, what can we say about the singular value decomposition of
$$\Sigma_\text{L} X \...
- 73
0
votes
0
answers
69
views
how to understand singular values geometrically like eigenvalues?
After a linear transformation, some vectors may not change direction, they only scale by a number. The scaling factor of those vectors is called eigenvalue.
Can we think of singular values in this ...
- 1
1
vote
0
answers
29
views
SVD distribution of linearly transformed Gaussian ensemble
The joint pdf of the singular values of the $m \times n$ Gaussian ensemble $X = x_{i,j} $, where the $x_{i,j} $'s are independent Normal(0,1) samples and the eigenvalues of the associated Wishart ...
- 11
0
votes
0
answers
96
views
Uniqueness of singular vectors (Theorem 4.1 Trefethen & Bau)
I am looking for some clarifications in the uniqueness portion of the proof of Theorem 4.1 of Trefethen & Bau's Numerical Linear Algebra.
The definition of the SVD and the proof exerpt from the ...
- 177
1
vote
1
answer
66
views
Bounds on the eigenvalues of $\bf W D W^\top$
Given a diagonal matrix $\bf D$ with diagonal entries $d_{ii} \in [0,1]$ and a matrix $\bf W$ with singular values $\sigma_i ({\bf W}) \in [0,1]$, can it be proven that the eigenvalues of $\bf W D W^\...
2
votes
1
answer
143
views
Singular values of uniform random points on hypersphere?
This question is motivated by a self-supervised learning problem in machine learning, but I'll try to strip out as many unnecessary details as possible. In this setting, we have large datasets and we ...
- 502
2
votes
1
answer
192
views
Singular value of identity minus 1-rank matrix
Given vectors $u,v\in \mathbb{R}^d$, I wonder what we can say about the minimum singular value of $I-uv^\top$? I know that when $u=v$, this matrix is symmetric so it is not hard to compute this. ...
- 115
1
vote
1
answer
30
views
Least nonzero singular value of $A^{-1} P$ for some invertible matrix $A$ corresponds to the reciprocal of the largest singular value of $A \circ P$?
Let $A$ be $n \times n$ real invertible matrix, and $P : \mathbb{R}^n \to \mathbb{R}^n$ be some orthogonal projection.
Then, what would be the relation between the smallest nonzero singular value of $...
- 7,923
0
votes
0
answers
104
views
Decide on near zero eigenvalues
I am using SVD to solve a homogeneous system of $N$ linear equations in $12$ variables, where $N \gg 12$, in the least-squares sense. In order to determine the null-space of the $N \times 12$ matrix, ...
- 31
1
vote
0
answers
94
views
SVD-based pseudoinverse solution sensitivity to equation linear combinations
My problem is of theoretical nature. Given an overdetermined system of $m$ equations in $n$ unknowns, $\bf A x = b$, where $m \gg n$ and
$$ {\bf A} = \begin{bmatrix} — {\bf a}_1 — \\ — {\bf a}_1 — \\ \...
- 11
1
vote
0
answers
42
views
When does a singular value of a square invertible matrix become an eigenvalue?
Let $A$ be an $n \times n$ invertible matrix with real entries. I know that its largest singular value, $\sigma_{\max}$ is equal to its operator norm. Moreover, there are singular vectors $u, v \in \...
- 7,923
0
votes
1
answer
118
views
Bounds on singular values of a Hankel-like matrix
Suppose I have a matrix where each successive row is a left-shift of the previous row (with a new value coming in on the right), e.g.,
$$A = \begin{bmatrix} 1 & 2\\ 2&3\\ 3&4 \end{bmatrix}...
- 11
2
votes
0
answers
39
views
Is the $k$-th elementary symmetric polynomial of the singular values the nuclear norm of the $k$-th alternating power of the matrix?
If $A$ is a complex $m$ by $n$ matrix, thus representing a $\mathbb{C}$-linear map from $\mathbb{C}^n$ to $\mathbb{C}^m$, we denote by $s_1, \ldots , s_r$ its singular values. Let $e_k$ denote the $k$-...
- 5,415
2
votes
0
answers
73
views
Maximum singular value via nonconvex QCQP
Finding the extremal singular triplet $(σ, u, v)$ of a generic real $m×n$ matrix $A$ can be formalized as a nonconvex quadratically-constrained quadratic program (QCQP):
$$σ = \max_{u,v}\quad u^⊤ A v ...
- 11.9k
1
vote
0
answers
88
views
Singular vectors of sums of outer products
I have a symmetric PSD matrix
$$ P = \sum_{i = 1}^N p_ip_i^\top \in \mathbb{R}^{n \times n} $$
where $p_i \in \mathbb{R}^n\ \forall i$. Its SVD is $P = U\Sigma_pU^\top$. I also have another sum
$$A = \...
- 33
1
vote
0
answers
64
views
Inequalities between biggest eigenvalue and singular values for a traceless matrix [closed]
Let $A \in {\Bbb C}^{4 \times 4}$ be a traceless matrix with Frobenius norm smaller than $1$. Let $\lambda_i$ be $i$-th largest (by modulus) eigenvalue of matrix $A$. Let $\sigma_i$ be $i$-th largest ...
