Questions tagged [svd]
In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix, with many useful applications in signal processing and statistics.
1,441
questions
0
votes
0
answers
16
views
Show $(XX')^{-1}= PD^{-2}P'$, where $X = PDQ'$ is a thin SVD
$X$ is $n \times p$ dimension matrix.
$X = PDQ'$ is a thin SVD, where $P$ is $n \times r$, $D$ is $r \times r$, and $Q$ is $p \times r$.
Here is what I tried:
$$(XX')^{-1} = (PDQ'QDP')^{-1} = (PD^2P')^...
- 303
1
vote
0
answers
34
views
How can one find the most suitable orthonormal basis for a dataset of complex numbers?
my first question here, and I am completely new to the game. I am an engineer, so pls don't hold it against me. I will go straight to the point.
Suppose I have a dataset of complex numbers (for ex, a ...
- 11
0
votes
0
answers
9
views
Matrix product decomposition
Consider a square real non-negative matrix $\mathbf{A}$.
My question is: is there a way to write $\mathbf{A}^{t}\big({\mathbf{A}^{t}}^{T}\big)$ (where $^T$ is transpose and $^t$ is matrix t_th power) ...
- 127
0
votes
1
answer
49
views
analysis of matrix multiplication via singular-value-decomposition? [closed]
Consider a square real matrix $\mathbf{A}$ and a real symmetric matrix of the same size $\mathbf{S}$
If I can diagonalise $\mathbf{A}$, such that $\mathbf{A}=\mathbf{B}\mathbf{D}\mathbf{B}^{-1}$, then ...
- 127
1
vote
0
answers
25
views
Maximum singular value via nonconvex QCQP
Finding the extremal singular triplet $(σ, u, v)$ of a generic real $m×n$ matrix $A$ can be formalized as a nonconvex quadratically-constrained quadratic program (QCQP):
$$σ = \max_{u,v}\quad u^⊤ A v ...
- 11k
0
votes
2
answers
82
views
Is every symmetric matrix in $\mathbb R^n$ a (non-uniform) stretch?
I conjecture that in $\mathbb R^n$, every symmetric matrix is a non-uniform stretch. Am I correct?
By non-uniform stretch, I mean that if $T$ is a non-uniform stretch, there exists an orthonormal ...
- 2,183
0
votes
0
answers
30
views
Singular vectors of sums of outer products
I have a symmetric PSD matrix
$$ P = \sum_{i = 1}^N p_ip_i^\top \in \mathbb{R}^{n \times n} $$
where $p_i \in \mathbb{R}^n\ \forall i$. Its SVD is $P = U\Sigma_pU^\top$. I also have another sum
$$A = \...
- 23
0
votes
0
answers
23
views
Maximizing inner product with an orthogonal matrix inside
Take given matrices $A, B \in \mathbb{R}^{m \times n}$. We want to find an orthogonal matrix $Q$ (in $\mathbb{R}^{n \times n}$) to maximize the inner product $\langle A, QB \rangle$.
So I know that if ...
- 1
0
votes
0
answers
27
views
SVD of a product by an $n \times r$ orthonormal matrix
Consider an $m \times n$ real matrix $M$ whose SVD is $U \Sigma V^\top$ and an $n \times r$ matrix $A$ with orthonormal columns.
If $r = n$ then the SVD of $M A$ is simply $U \Sigma \left( A^\top V \...
- 11
0
votes
1
answer
36
views
SVD: Picking U and V when singular values are repeated
Is there a correct/stable way of dealing with repeated eigenvalues in S?
I was working on a Moore-Penrose Inverse in a library for a programming language at work. The dummy matrix I picked turned out ...
- 21
3
votes
1
answer
51
views
Relating the SVD of $AB^T$ and $B^TA$
Let $C$ be a $m \times n$ real matrix. We can show that $CC^T$ and $C^TC$ have the same non zero eigenvalues and their respective eigenvectors can be related through $Cv = \lambda^{-1/2} u$, $C^Tu = \...
- 503
0
votes
0
answers
31
views
Finding an orthonormal basis of $\mathbb C^3$ so that matrix representation consists of blocks of a certain form
Problem: Suppose $A$ from $\mathbb{C}^3$ to $\mathbb{C}^3$ is given by
$$
A=\left[\begin{array}{ccc}
0 & -1 & 2 \\
1 & 0 & -3 \\
-2 & 3 & 0
\end{array}\right].
$$
Question: How ...
- 693
2
votes
1
answer
83
views
Diagonally dominant behavior for SVD decompositions
Whenever I work with matrices for the SVD decomposition in the following manner:
X = np.random.randn(100,20)
...
- 862
1
vote
2
answers
87
views
SVDs of the matrix $\begin{bmatrix} 1 & 1 \\ 0 & 0\end{bmatrix}$
This is from Trefethen's Numerical Linear Algebra, lecture 4, exercise 1(d).
Determine the SVDs of the matrix
$$\begin{bmatrix} 1 & 1 \\ 0 & 0\end{bmatrix}$$
This matrix maps both $[1, 0]$, $...
- 1,041
0
votes
0
answers
13
views
Given Gaussian "noise" matrix $G$ and matrix products $AG$, and $A^\intercal AG$, solve for $A$
Given Gaussian "noise" matrix $G$ and matrix products $AG$, and $A^\intercal AG$, solve for $A$
Let $A \in \mathbb{R}^{m \times n}$ be a matrix with rank $k$. Consider the following ...
- 2,519
3
votes
0
answers
23
views
Eigenvalue decompositin equal to singular value decomposition
Question: when is the eigenvalue decomposition of a matrix equal to its singular value decomposition?
Answer: when A is hermitic and has positive eigenvalues.
I don't really understand. If we have A=A*...
- 161
0
votes
1
answer
32
views
Show that a non invertible matrix $B$ cannot exist for $||A\vec x - B\vec x || \le 1$
Let $A$ be a $5\times 5$ matrix with singular values $10,10^2,10^3,10^4,10^5$. Show that there cannot exist a non-invertible matrix $B$ such that $||A\vec x - B\vec x|| \le $1 with $||\vec x|| = 4$.
...
- 185
1
vote
0
answers
27
views
Fast algorithm for incremental randomized SVD
I have a list of covariance matrices $\{\Sigma_i\}$. I want to be able to take the (randomized for performance) SVD of the average of different (incremental) subsets of this list in order to perform a ...
2
votes
1
answer
87
views
Bound of $\lVert \mathbf{Ax} \rVert$ with $\mathbf{AA}^\top=\mathbf{I}$
The original question (informal):
Consider the matrix $\mathbf{A}\in\mathbb{R}^{M\times N}~(M\le N)$ such that $\mathbf{AA}^\top=\mathbf{I}_M$. I want to determine whether the value of $\lVert \mathbf{...
- 598
0
votes
1
answer
61
views
How we can proof SVD : $M = U\sigma V^T = \sum_{i=1}^r \sigma_i u_i v_i^T $
How we can prove in SVD this equation: $M = U\sigma V^T
= \sum_{i=1}^r \sigma_i u_i v_i^T
= \sigma_1 u_1 v_1^T + \sigma_2 u_2 v_2^T + \cdots + \sigma_r u_r v_r^T$?
I don't see it...
I know that:
The ...
- 1
0
votes
0
answers
25
views
Calculating Randomized Singular Values has two main steps
I am learning how calculating Randomized Singular value works from a text book.
They describe it in 3 steps. Let's say $X \in R^{m \times n}$ and we'd like to find a randomized svd for it.
We believe ...
- 65
0
votes
1
answer
84
views
If $M\in M_{m \times n}(\mathbb{R})$ is a rank $r \geq 1$ matrix then it can be written as a sum of exactly $r$ rank 1 matrices
I have problem with the proof of the following statement.
If matrix $M\in M_{m \times n}(\mathbb{R})$ has rank $r \geq 1$, then it can be written as a sum of exactly $r$ rank-$1$ matrices
I try in ...
- 11
0
votes
0
answers
12
views
Projection of $W$ onto the subspace of $\triangle W$?
In the LoRA paper (https://arxiv.org/pdf/2106.09685.pdf), they stated in H.4 that:
One can naturally consider a feature amplification factor as the ratio $\frac{\|\Delta W\|_F}{\left\|U^{\top} W V^{\...
0
votes
0
answers
24
views
Constrained low-rank matrix approximation
I recently came across the low-rank matrix approximation method: a method to identify the “best” way to approximate a given matrix $A$ with a rank-$k$ matrix $A_k$, using singular value decomposition (...
- 21
0
votes
0
answers
14
views
Is it valid to rely on a derived matrix for interpolation/extrapolation?
For example, suppose you have a known $\mathbf{A}$ and a known $\mathbf{Y}$ with an unknown $\mathbb{X}$:
$$
\mathbf{A}\mathbb{X} = \mathbf{Y}\\
\mathbb{X} = \mathbf{A^{-1}Y}
$$
Now, suppose you ...
- 488
0
votes
1
answer
36
views
SVD decomposition of block matrix
Given block matrix $B = \begin{pmatrix} 0 & A^{T} \\ A & 0\end{pmatrix}$ and matrix $A$ has certain SVD decompostion: $A = VDU^{T}$. My goal is finding SVD decomposition of matrix B, using ...
- 31
1
vote
1
answer
41
views
Proof check that $\|A\|_{F}^{2} = \sigma_{1}^{2} + \sigma_{2}^{2} + \dots + \sigma_{k}^{2}$
Consider $\|A\|_{F} = \sqrt{\langle A, A\rangle)}$ where $\langle X, Y\rangle = \text{Tr}(X^{*}Y)$ where '*' denotes the conjugate transpose of a matrix.
I was asked to prove the following question:
...
- 63
1
vote
1
answer
41
views
singular value decomposition of sum
Let $A,B$ be positive, linear trace class operators on some Hilbert space.
I would like to know if the following trace inequality for some $\mu>0$ is true
$$
\mathrm{Tr}\!\left(A\,(A+B+\mu I)^{-1}\...
- 145
1
vote
0
answers
46
views
Does SVD work for non-square matrices?
I'm trying to decompose a matrix of keypoints for Tomasi-Kanada factorization
I am not doing this by hand right now simply because there are libraries available for this and I know I will get the ...
- 111
0
votes
0
answers
25
views
SVD: intuition of why linear transformation takes some orthogonal basis to orthogonal vectors
The singular value decomposition (SVD) was introduced to us as $MV = U\Sigma$, i.e., finding some mutually orthogonal normalized vectors (columns of $V$) that map to orthogonal vectors $U$ (then we ...
- 221
0
votes
0
answers
18
views
SVD of matrix of eigenvectors
Let $A$ be a (complex) matrix with eigendecomposition $A = U S U^{-1}$, where $U$ is the invertible matrix of eigenvectors.
What is known about the singular value decomposition (SVD) of $U$?
- 181
0
votes
1
answer
58
views
Can we use the singular value decomposition to compute the matrix exponential for a non-diagonalisable matrix?
For a diagonalisable matrix $ \bf{A} $ with eigendecomposition $ \bf{A} = \bf{U} \bf{\Lambda} \bf{U}^{-1} $, we know that $ \exp(\bf{A}) = \bf{U} (\exp \bf{\Lambda}) \bf{U}^{-1} $, where $ \exp \bf{\...
- 180
0
votes
0
answers
53
views
Connection between ratio of magnitude of eigenvalues and ratio for singular values
Let $A$, $B$ be a square, complex $4 \times 4$ matrices such that $Tr(A) = Tr(B) = 0$ Let $s_i(X)$ be i-th biggest singular value of matrix X. Let $\lambda_i(X)$ be i-th biggest eigenvalue (by ...
0
votes
0
answers
28
views
Singular Value Decomposition(SVD) of sum of two Hermitian matrices.
Suppose the Singular Value Decomposition(SVD) of an $m \times n$ matrix $A=U_1S_1V_1^H$ where $V_1^H$ denotes the Hermitian conjugate of $V_1$. Then the SVD of $AA^T=U_1S_1V_1^HV_1S_1^HU_1^H=U_1S_1^...
- 25
0
votes
0
answers
33
views
A proof of permutation matrix.
For a square matrix $A \in \Bbb R^{n\times n}$, if its singular values are identity matrix $I$, moreover, the summation of each row is equal to $1$, and all elements are non-negative, can we prove ...
- 3
0
votes
0
answers
20
views
How does the usual algorithm for SVD matrix factorization work?
I get that SVD is a way of representing an arbitrary linear transformation on ${R}^n$ as a composition of a scaling, a rotation, and a reflection transformation, and I've seen plenty of insightful ...
1
vote
0
answers
34
views
Why are there two solutions to this SVD decomposition problem?
I was faced with the task of decomposing the following matrix via SVD, i.e. decomposing the matrix into the form $USV^T$.
\begin{align*}
A = \begin{bmatrix}3&2&2\\2&3&-2\end{bmatrix}
...
- 65
0
votes
0
answers
27
views
Working with change of coordinates in singular value decomposition (SVD)
I am given a matrix A which I need to take the SVD of. Currently I have found the vectors U, Σ and V such that A=UΣV. As the next part of my task I need to find the matrix A in V coordinates. To solve ...
0
votes
0
answers
29
views
Working with singular value decomposition (SVD)
I am given a matrix $A$ which I need to take the SVD of. Currently I have found the vectors $U$, $\Sigma$ and $V$ such that $A = U \Sigma V$. As the next part of my task I need to find the matrix $A$ ...
- 1
-1
votes
1
answer
88
views
Simplify ${\rm trace}((A_1⊗A_2+B_1⊗B_2)^{-1})$
This equation is take a long time in simulation due to Kronecker product ($⊗$).
Simplify $${\rm tr}((A_1⊗A_2+B_1⊗B_2)^{-1})$$
where tr is trace operator (sum of diagonal elements).
Also, $A_1,A_2,B_1,...
- 101
2
votes
1
answer
56
views
Eigendecomposition of $A^TD_iA$, with $A$ rectangular and $D_i$ diagonal
Given $A^TD_iA$, where:
$A$ is a rectangular matrix
$D_i$ is a diagonal matrix with positive entries
$A^TD_iA$ is a positive definite matrix
I need to compute the eigendecompositions of $A^TD_iA$, ...
- 55
1
vote
1
answer
71
views
Can SVM be special case of PCA?
Let $X$ and $Y$ two linearly separable finite subsets of a $K$-dimensional real vector space $V$ with orthonormal basis $A = \{a_1,\ldots, a_K\}$. The covariance matrix $\Sigma_A$ of the set $X \cup Y$...
0
votes
0
answers
10
views
A singular value decomposition of PAQ^* in terms of a SVD of A
Let A be an m x n matrix, P a unitary m x m matrix, and Q a unitary n x n matrix. How do you write an SVD of PAQ* in terms of an SVD of A?
Is this as simple as I think? like A=U$\Sigma$V*, so P(U$\...
- 11
0
votes
0
answers
27
views
SVD of $A$ has rank $r$. How to write $A$ as a sum of $r$ rank-$1$ matrices? [duplicate]
Suppose $A = U \Sigma 𝑉^*$ is the SVD of $A$, and $A$ has rank $r$. Using this decomposition, how do I write $A$ as a sum of $r$ rank-$1$ matrices?
I am very confused about how to approach this. I ...
- 11
1
vote
0
answers
50
views
Uniqueness of $UV^T$ of non-singular matrix $A$
Let $A$ be a $n \times n$ non-singular matrix. Then we can take many alternative SVD from from $A$. Like $A = U_1 \Sigma V_1^\top = U_2 \Sigma V_2^\top = \cdots$ where $\Sigma$ is an ordered diagonal (...
- 11
0
votes
1
answer
24
views
What is the singular value of such a simple permutation matrix?
The matrix $\left[\begin{array} &0 &1\\1& 0\end{array}\right]$
The following two solutions give different answers:
The first |A-λI|=0
(-λ) 1
1 (-λ)
= 0
∴(-λ)×(-λ)-1×1=0
∴(λ2)-1=0
∴(λ2-...
- 3
0
votes
0
answers
49
views
Why does subtracting a rank-$1$ matrix shift the singular vectors?
To compute SVD of a matrix: Say you start with the matrix $A$ and you compute $v_1$. You can use $v_1$ to compute $u_1$ and $\sigma_1(A)$. Then you want to ensure you're ignoring all vectors in the ...
- 812
0
votes
0
answers
21
views
Perturbation bound for singular vectors
Given an adjacency matrix $A={A^T}\in {\mathbb{R}^{n \times n}}$ of a simple undirected graph and its degree matrix $D$ .
When I add at most $Q$ edges into the graph, which is equivalent to adding a $...
- 11
-1
votes
1
answer
37
views
Eigenvalues Bounded by Singular Values Proof
I'm trying to prove the inequality for a real square matrix $M$
$\sigma_\min(M)\leq|\lambda_i|\leq\sigma_\max(M)$ for all $i$
We have that:
$\lambda_\max(M)=\sup_{x\neq0}\frac{x^TMx}{||x||_2^2}$
$\...
- 13
1
vote
1
answer
98
views
Why the singular values in SVD are always hierarchical/descending?
Please, I'm trying to understand why singular values (SV) are always hierarchical/descending. At the beginning of my studies, I thought that the hierarchy of sigmas ($ \sigma_1 \geq \sigma_2 \geq ... \...
- 19