In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix, with many useful applications in signal processing and statistics.

learn more… | top users | synonyms

0
votes
0answers
21 views

SVD proof to If $A$ is of full rank, then $A^{*}A$ is of full rank

Provided $A$ is a full rank matrix $\in\mathbb{C^{m\times n}}$, then $A^{*}A$ is of full rank. Suppose $m\gt n$. There is a solution to this problem: solution link, and the top solution makes ...
3
votes
0answers
21 views

Merging two SVD factorizations but using only $U_1S_1$ and $U_2S_2$ (question about Radim's Phd Thesis [gensim])

I am reading Radim's Phd Thesis (the creator of gensim), on the chapter about SVD (page 45). There he puts certain assignment, indicating that we can calculate the "merge" of a couple of SVD ...
0
votes
0answers
13 views

SVD of partitioned matrix where all cells except one are zero

Let $A$ be real valued matrix of size $n \times n$. Let the SVD of $A$ be $$A= UDV^T.$$ I am interested in $$Q=VU^T.$$ Now assume we expand $A$ with zero rows and columns to get the block matrix ...
0
votes
0answers
10 views

Singular value decomposition - unique determination

i am self-studying SVD - and stumbled upon the Wikipedia page (https://en.wikipedia.org/wiki/Singular_value_decomposition) on the statement that a common convention is to order the singular values in ...
1
vote
1answer
29 views

How do the rows of a change of basis matrix form a basis for expressing columns?

I am reading this article on Principal Component Analysis (PCA) and in section III-B (page 3) it has strange definition I don't understand. In the toy example $\mathbf{X}$ is an $m \times n$ ...
0
votes
1answer
14 views

Does SVD always produce eigenvectors which are normal

As per the title heading, I have always been using SVD on covariance matrices to find the principal component for a set of data points; but I was wondering if I were to put in another matrix, will ...
1
vote
0answers
18 views

Singular value decomposition for complex Hermitian matrix

I am computing singular value decomposition on a covariance matrix with complex values (goal: principal component analysis). I read here that it is possible to use svd algorithms for real values using ...
1
vote
0answers
13 views

SVD of Block-Partioned Hankel matrices

I am trying to do SVD of a large block-hankel matrix for model order reduction (Low rank approximation). However, I quickly run into memory issues in forming the large Block-Hankel matrix and CPU ...
3
votes
2answers
46 views

rank 1 constraint

How can I find a scalar $s$ that makes the matrix $A+sB$ have rank $1$, where $A$ and $B$ are $3\times 3$ matrices? Is there a method using singular value decomposition or eigenvalues? Thanks!
0
votes
0answers
25 views

Linear Algebra: Eigenvalues of matrix multiplied with transpose.

In SVD, the orthogonal matrices $U$ and $V^T$ have same nonzero eigenvalues in their complete set of eigenvalues. What might the proof of this statement? The cited document is - here. On page 20.
1
vote
2answers
43 views

Does spectral norm of a square matrix equal to its largest eigenvalue in absolute value?

I have one simple question. Given the spectral norm $\left \| . \right \| _2$ of a matrix $A$, which is equal to the squareroot of the largest eigenvalue of $A^{^*}A$ $$\left \| A \right \| ...
0
votes
0answers
13 views

how to calculate SVD on a matrix incrementally

I know how to calculate SVD if I could load the whole matrix into memory, but sometimes I couldn't do that because the matrix might too large to fit into memory. For example, I have a matrix $$A = ...
1
vote
0answers
31 views

Numerically stable SVD

In this question regarding SVD, it is explained why eigen decomposition of $ A^tA $ is not numerically stable compared to "direct SVD algorithms". Since the former is the algorithm I'm most familiar ...
2
votes
0answers
19 views

Singular value decomposition about complex matrix

I am now stuck on the following problem. Could anyone help me out? Suppose a rank $1$ $n \times n$ matrix $A$ is form as a follows, $A=a*a'$, were $a$ is a $n\times 1$ vector. Now, suppose we have ...
0
votes
1answer
16 views

Relationship between eigenvectors and singular vectors of a Hermitian matrix?

What is the relationship between the eigenvectors and singular vectors of a Hermitian matrix? Intuitively, I would expect them to be the same (modulo scaling). However, this doesn't seem to be the ...
0
votes
0answers
23 views

Constrained SVD

I have a matrix $A$, and I can run svd on it to get $UDV'$. Of course, the $U$ and $V$ are not unique, and I'm looking for a particular pair of $U$ and $V$. I want to find $U$ and $V$ such that the ...
0
votes
0answers
10 views

Singular Value Decomposition / Principal Component Analysis - relevance of different matrices

I am trying to conceptually understand SVD. So, if the matrix decomposition is: A = USVT what does VT represent? I get what U ...
0
votes
1answer
18 views

Matrix multiplication error or is it just me?

I was reading a bit on SVD and ran into this "error" http://web.mit.edu/be.400/www/SVD/Singular_Value_Decomposition.htm
3
votes
1answer
49 views

SVD products and the approximation error using Frobenius norm

This is my first time posting a question here. The question is about quantifying the bound of error of SVD products using the Frobenius norm. Suppose I have a rectangular & real matrix $A (m ...
1
vote
0answers
50 views

Can I clamp singular values of $3\times3$ matrix without effectively computing SVD?

I have a $3\times3$ matrix $A$, and compute its SVD $U \Sigma V^\star = A$. I clamp the singular values in $\Sigma$ to some small range (e.g. $[0.5, 1.5]$ ) and reconstruct matrix $\widetilde{A}=U ...
0
votes
0answers
28 views

the SVD (singular value decomposition) of an augmented matrix

Suppose we have a $4\times 3$ dimensional matrix $A$. Denote the SVD of $A$ by $USV^T$, where $U\in R^{4\times 3}, S\in R^{3\times 3}, V\in R^{3\times 3}$. Then, we construct a new matrix $B=[A;0]\in ...
1
vote
1answer
36 views

Using SVD to approximate matrix-vector multiplication?

Given some matrix A, is it possible to use Singular Value Decomposition to approximate Ax for some vector x within some error bound? According to Efficient low rank matrix-vector multiplication, it ...
0
votes
0answers
24 views

advantage and disadvantage of using SVD to solve least square problems

I usually just use $AA^T$ or QR decomposition of A to solve least square problems. But SVD seems to be the popular way to solve the problem. what is the advantage and disadvantage of SVD? thanks!
2
votes
0answers
43 views

Largest Singular Value / Singular Value

I was wondering, what if the eigenvalues of a matrix A are all negative. So does that simply mean there is no singular value for this particular matrix?, hence I can't calculate the conditional number ...
0
votes
0answers
37 views

Fundamental Theorem of Linear Algebra and SVD

Here is the pdf I am using to learn SVD Decomposition. On page $2$, it states how the eigenvectors from non-zero eigenvalues of $A^{T}A$ are in the rowspace of $A$ and how eigenvectors from non-zero ...
0
votes
0answers
26 views

a simple question about singular value decomposition(SVD)

This question have made my mind busy for a long time. Assume $X=X_1+X_2$ where $X$ is a $m \times n$ matrix and $X_1 \perp X_2$. Then how can I connect $SVD_X$ to $SVD_{X_1}$ and $SVD_{X_2}$. As I ...
1
vote
0answers
16 views

Simplify product (Hadamar / Kronecker) of matrices for singular value decomposition in Demmler-Reinsch Orthogonalization

Let $1_b = (1, \ldots, 1)^\prime$ be the $b$-dimensional vector of ones, let $\circ$ and $\otimes$ denote the Hadamar respectively the Kronecker product and let $n = m \cdot c$, where $n,m,c \in ...
0
votes
0answers
49 views

Finding $Q$ for any $A$ s.t. $QAQ^\top = I$

Given an invertible and PSD matrix $A$, I am looking to find $Q$ such that: $$ QAQ^\top = I $$ What is a/the right/efficient way to do this? Here is what I did: SVD gives $$ A \approx U S V^\top ...
0
votes
0answers
29 views

How close is Cartesian product of unit orthogonal bases of SVD to identity matrix?

If I have N unit orthogonal vectors of length N $\phi_{i,N\times 1}$ obtained from SVD of a $N\times M$ matrix $U$ : $$ U_{N\times M} = \sum_i^N \sigma_i\phi_{i,N\times 1}\times\psi_{i,1\times M}\\ ...
1
vote
0answers
22 views

SVD of a block partitioned matrix

Given a block partitioned matrix $\boldsymbol{A}$ $$ \boldsymbol{A} = \begin{bmatrix} \boldsymbol{A}_{1,1} & \boldsymbol{A}_{1,2} & \cdots \\ \boldsymbol{A}_{2,1} & ...
0
votes
0answers
11 views

General formula for $\hat{b}$ (least squares) using SVD and pseudoinverse

In a situation with an SVD for A given by $A=U\Sigma V^T$ I know about the relation $ x=(A'A)^{-1}A'b=A^+b $ Given b and matrix A, which general formula can one use to find $\hat{b}$?
0
votes
1answer
28 views

Solve Ax=0 using Single Value Decomposition

Trying to solve Ax=o when $A=\begin{bmatrix}2&1&-1\\1&2&1\\ \end{bmatrix}$ using single value decomposition. I have the s,v,u and was thinking that x was as simple as $x=s*s^t$ but ...
0
votes
0answers
33 views

Finding the largest singular value “easily”

Im only interested in finding the largest singular value. I don't need the singular vectors. Is there a way to do so without performing full SVD? Is there an analytical expression? If not, is ...
1
vote
0answers
19 views

SVD: How to find the columnvector of U corresponding to a singular value equal to zero

The question is if you have a situation where one of the singular values is equal to 0 in a singular value decomposition of a matrix, how to do you procede to find the column vector of U corresponding ...
0
votes
0answers
21 views

SVD and least squares solution: orthogonal projection $ \vec{\widehat b} $ of $ \vec{b} $ onto $Col(A^T)$

Given the following: $ A = \left(\begin{array}{rrr} -2 & 3 & 2 \\ 2 & 2 & 3 \end{array}\right).$ A has the SVD: $A = USV^T$ $ b = \left(\begin{array}{rrr} -6\\ 1 \\ 4 ...
0
votes
1answer
22 views

Simplifying the inverse of the sum of 2 matrices

I would like to simplify the following inverse computation : $$(D+A)^{−1}$$ where $A=UΣU^T$ (eigenvalue decomposition). And D is a diagonal matrix such that $D = \lambda \boldsymbol{I}$ I know the ...
0
votes
1answer
30 views

inverse of sum of diagonal matrix and eigendecomposition

I would like to simplify the following inverse computation : $$(D + A)^{-1}$$ where $A=U\Sigma U^T$ (eigenvalue decomposition). And D is a diagonal matrix I know the inverse of A is ...
2
votes
0answers
59 views

Restoring Bidiagonality to a Matrix in SVD Algorithms

Good Afternoon, I am implementing the Golub-Reinsch SVD algorithm and am having difficulty with a boundary case Given a bidiagonal matrix of the form: $$ \begin{bmatrix} b11 & ...
1
vote
1answer
54 views

Find unit singular vectors for two known singular values.

$$A=\begin{pmatrix}-3 & -1 \\ -3& 1\end{pmatrix}$$ Find the singular values of $σ_1$ and $σ_2$, Find unit vectors $v_1$ and $v_2$ such that $||Av_1|| =σ_1$ and $||Av_2|| =σ_2$ I figured ...
2
votes
1answer
54 views

SVD and transpose of a skinny matrix

Show: If $\mathbf{A}\in\mathbb{R}^{M\times N}$ with $M\geq N$, then there exists a matrix $\mathbf{G}$ with orthonormal rows so that $\mathbf{A}^T=\mathbf{G}\mathbf{A}\mathbf{G}$. I'm pretty lost on ...
0
votes
0answers
22 views

how to solve this low rank approximation problem

$$L = \sum_{i=1}^n (N_i-C_i)^T\Sigma^{-1}(N_i-C_i) =Tr((N-C)\Sigma^{-1}(N-C)^T)$$ $$=\sum_{i=1}^n \sum_{j=1}^d (N_{ij}-C_{ij})^2\sigma_j^{-2}$$ $$rank(C)=k<rank(N)$$ Basically i need to find the ...
1
vote
2answers
32 views

Why $||U\Sigma V^\intercal - X_k||_F = ||\Sigma - U^\intercal X_k V ||_F$

Here $U\Sigma V^\intercal $ is the svd decomposition of M.I know that U and V are othogonal but i dont know which property it is using. This in reference to the post Proof of Eckart-Young-Mirsky ...
0
votes
0answers
33 views

minimization of weighted frobenius norm for pca

So my problem is i like to derive pca solution as the maximum likelihood estimate for the true data.So basically i am assuming that my measured data has two component one is low rank component and ...
1
vote
1answer
88 views

vector as linear combination of other vectors with one more perpendicular vector

I am reading about Singular Value Decomposition (SVD) from book SVD CSTheory Infoage. At page 6, the chapter says: A matrix $A$ can be described fully by how it transforms the vectors $v_i$. Every ...
1
vote
2answers
26 views

Compactness argument in SVD existence proof

The classical proof of the existence of the SVD factorization by Trefethen and Bau reports Set $\sigma_1 = \mid\mid A \mid\mid_2$. By a compactness argument, there must be a vector $v_1 \in ...
0
votes
0answers
30 views

Singular value decomposition positive components

I am using Singular Value Decomposition (SVD) applied to Singular Spectrum Analysis (SSA) of a timeseries. ...
0
votes
2answers
26 views

If rows of matrix M are switched, do the singular values of M change?

I have looked for a source on the relationship between elementary matrix row operations and singular values, but I can't find a good, compact set of information. I'm really only interested in ...
2
votes
1answer
56 views

Efficiently compute the eigenvectors of the Laplacian of a symmetric positive matrix

I am working with a matrix A relatively large (200k x 200k), and I want to compute the eigenvectors of the Laplacian: $L = D - A^2$, where $A$ is symmetric. I don't need all eigenvectors, just a few ...
0
votes
2answers
153 views

Changes in singular values of matrix when rows are added

I know that if a column is added to a matrix then the matrix largest signular value increases and the smallest singular value decreases. That is: Given matrix $A \in R^{m \text{x} n}$, $m>n$, and ...
0
votes
0answers
25 views

Significance of minimum singular value

I came across this statement from Wikipedia on singular value decomposition. A total least squares problem refers to determining the vector $x$ which minimizes the 2-norm of a vector $Ax$ under ...