In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix, with many useful applications in signal processing and statistics.

learn more… | top users | synonyms

0
votes
0answers
10 views

Singular values of the product of two (semi)orthogonal matrices

Let's assume we are given two (semi)orthogonal matrices $U_1$ and $U_2$ with dimension $m\times n$ such that $m>n$. The (semi)orthogonality means $$U_1^TU_1 = I_n$$ and $$U_2^TU_2 = I_n$$ but ...
0
votes
1answer
16 views

Proof that left singular vectors in SVD are orthogonal, and proof of low-rank approximation

I've been reading about SVDs and have a couple questions. First, let $A\in \mathbb{R}^{n\times d}$ be a matrix with SVD $U\Sigma V^T$. Let $\sigma_i$ denote the $i$'th singular value, with ...
2
votes
1answer
18 views

Compute the SVD of $AB$ from the SVDs of $A$ and $B$

Knowing the SVD of $\mathbb{C}^{m*n} \ni A = U_A\Sigma_AV_A$ and $\mathbb{C}^{n*s} \ni B = U_B\Sigma_BV_B$, is there any way to speed up the calculation of the SVD of $AB = U_{AB}\Sigma_{AB}V_{AB}$? ...
0
votes
0answers
15 views

Given A $m\times n$, $\{u_1,…,u_n\}$ ON basis of $R^n$, prove eigenvalues aren't negative [duplicate]

Given $A$ $(m\times n)$, $\{u_1,...,u_n\}$ ON(orthonormal) basis of $R^n$ which are eigenvectors of $A^TA$ with $\lambda_1 , ... , \lambda_n$ eigenvalues accordingly. Prove: Eigenvalues are not ...
2
votes
3answers
173 views

Minimizing $\|Ax\|_2$ subject to $\|x\|_2 = 1$

I have a Matlab program to estimate a vector $x$ from noisy measurements. I use the singular value decomposition (SVD) to solve the linear equation $Ax=0$ (where the number of equations is greater ...
1
vote
1answer
38 views

Find SVD of $A$

How do I find the singular values? They somehow show that $\lambda_1 = 27, \lambda_2 = 6, \lambda_3 = 0$. I still can't see how they found them with the equations I made in my solution.
1
vote
1answer
22 views

invertible subspaces and SVD

Suppose X is a $m\times d$ real matrix. I am trying to prove the following claim: $XX^T$ is invertible iff $ span\{x_1,x_2,...,x_m\}=\mathbb{R}^d $ such that $x_1, x_2, ..., x_m $ are X's columns. ...
1
vote
1answer
30 views

low rank approximations and diagonalization

I would like to discuss or hear an opinion about the following. Given is the (hermitian) $n\times n$ matrix $A = D+M V M^{\dagger}$ with D diagonal. I would like to calculate the eigenvalues (and ...
4
votes
1answer
79 views

Prove that $A^2=A\iff \Sigma K=I_r$

Let $A$ be a square complex matrix and let $A=U\Sigma V^*$ be a singular value decomposition. Then $A$ can be written as $$A=U\begin{bmatrix} \Sigma K & \Sigma L\\ 0 & 0 \end{...
1
vote
0answers
16 views

Efficient distributed algorithm for Singular Value decomposition

Is there any distributed algorithm possible for calculating Singular value decomposition of a matrix? As far as if have explored at many places (like here pg-132) it is written that because of high ...
0
votes
1answer
44 views

why bother with extra orthonormal vector in Singular value decomposition

when we do the SVD for a $m\times n$ matrix, we have to extend the set $u_1, ... , u_r$, to an orthonormal basis $u_1, ... , u_m$ for $R^m$ if $r<m$. But why don't we just fill zero vectors to make ...
3
votes
0answers
39 views

SVD as a solution to linear least squares

I'm a little confused about the various explanations for using Singular Value Decomposition (SVD) to solve the Linear Least Squares (LLS) problem. I understand that LLS attempts fit $Ax=b$ by ...
2
votes
1answer
35 views

Orthogonalise a matrix with respect to another matrix

I need to do a particular operation and I am not able to find a name or some ideas to make the operation happen. I have a matrix $F_s \in \mathbb{R}^{n \times n_s}$. I do an Singular value ...
0
votes
1answer
47 views

SVD and Low-rank approximation

In the proof of Low-rank approximation by Trefethen & Bau, It is written: Theorem 5.8 : A is an $m \times n$ Matrix. For every $v$ with $0 \leqslant v \leqslant r$, define $$ A_{v}=\...
0
votes
0answers
11 views

Independent components with a known relationship between sources

I'm performing an experiment, and I expect that my results of the form: $$\mathbf{M}=\mathbf{ce}, \mathbf{M}\in\Bbb{R^{m\times n}},\mathbf{c}\in\Bbb{R^{m\times 2}},\mathbf{e}\in\Bbb{R^{2\times n}}$$...
1
vote
1answer
104 views

Calculating SVD By Hand

When calculating the SVD of the matrix $$A = \begin{bmatrix}3&1&1\\-1&3&1\end{bmatrix}$$ I followed these steps $$A A^{T} = \begin{bmatrix}3&1&1\\-1&3&1\end{bmatrix} ...
0
votes
0answers
18 views

Clarification between the left and right singular vectors of the singular value decomposition (SVD).

I am hoping for clarification on the difference between the left and right singular vectors of the SVD for some matrix: $A = U\sum V^T$. I'd like to explain how I understand these vectors using the ...
0
votes
0answers
26 views

Question on SVD Uniqueness Proof

I have problem understanding the proof of uniqueness for SVD by by Trefethen & Bau. If the lengths of the semi-axes of the hyper-ellipse are distinct, then semi-axes themselves are ...
1
vote
0answers
41 views

Optimal ordering in Jacobi SVD algorithm

In Jacobi SVD algorithm as given here every pair of columns of the matrix is orthogonalized until convergence. I want to know that how does the order of selection of the pair of columns affect the ...
0
votes
1answer
25 views

Find spectral theorem of $A$ and find its singular eigenvalues.

The rotation matrix $$A=\pmatrix{ \cos \theta & \sin \theta \\ -\sin \theta & \cos \theta}$$ has complex eigenvalues $\{e^{\pm i\theta}\}$ corresponding to eigenvectors $\pmatrix{1 \\i}$ and $...
0
votes
0answers
15 views

SVD matrix properties after blockwise scalar multiplication

Are there any findings relating the singular value decomposition (SVD) of a block matrix to the SVD of the same matrix after multiplying each block with a constant ? In other words, let $A \in \...
4
votes
2answers
48 views

SVD decomposition

I'm currently looking for the SVD decomposition of $$\begin{bmatrix} 0 & 1 \\ 0 & 0 \\ 0 & 1 \end{bmatrix}$$ but I am having a struggle, because $A^TA$ has a ...
0
votes
0answers
14 views

Extending $\{u_1, u_2\}$ to an orthonormal basis when finding an SVD

I've been working through my linear algebra textbook, and when finding an SVD there's just one thing I don't understand. For example, finding an SVD for the 3x2 matrix A. I will skip the steps of ...
2
votes
1answer
25 views

Element with maximum magnitude in $A \leq \max(\sigma_{i})$, where $\sigma_{i}$ is singular values of A

Let $A$ be a matrix with real values. Is it true that element with maximum magnitude in $A$ is less than $\max(\sigma_{i})$, where $\sigma_{i}$ is singular values of A? That is, is $$ \max_{ij} |A_{ij}...
0
votes
0answers
10 views

constrain general solution of ill-posed linear system to $\Re_+$?

I have a solution space of an under-determined linear system Ax = b with n x m matrix A: $$x= x0 + V2 * c (1)$$ with [U, S, V] = svd(A); V2 = V(:,r+1:end); $x0 = A^+ b; $ r = rank(A); I ...
0
votes
1answer
16 views

square eigenvectors for Singular Value Decomposition?

This is from my textbook What I don't understand is, $V$ and $U$ are already square, why the textbook says "if we want to make them square"?
0
votes
1answer
27 views

Rank of product of SVD vectors

Given a matrix $M$, we can compute its singular value decomposition $M=U\Sigma V^*$ where $^*$ is the complex conjugate transpose. $U$ and $V$ are unitary, so $UU^*=I$, $VV^*=I$. Let's take the $i$-th ...
1
vote
0answers
10 views

Perturbations on SVD left singular matrix

Given a symmetric matrix $A\in \mathbb{R}^{n\times n}$, with all the entries greater than zero $A_{i,j}>0$ with rank $k<n$, we can calculate its SVD decomposition: $$ A = USU' $$ Assuming now ...
1
vote
0answers
24 views

Jacobi SVD algorithm implementation

Is this implementation of Jacobi SVD algorithm according to the standard algorithm? Please verify. Is this Hestenes Jacobi method? I have seen pseudo code of Jacobi algorithm like here which ...
0
votes
0answers
42 views

null space from SVD

It is said that a matrix's null space can be derived from QR or SVD. I tried an example: $$A= \begin{bmatrix} 1&3\\ 1&2\\ 1&-1\\ 2&1\\ \end{bmatrix} $$ I'm convinced that QR (more ...
0
votes
1answer
28 views

Possible quick solution of SVD of covariance matrix of Xv, where v may change, while X does not.

I am current trying to work on one algorithm, that for Iteration $t$, I need to calculate the SVD of $(X\text{diag}(v^t))^T(X\text{diag}(v^t))$. This could be very slow if $X$ is of high dimension. ...
0
votes
1answer
21 views

SVD and homogeneous equation

Suppose a $m \times n$ matrix $A$, and column vector $h$. ($A$'s rank is equal or smaller then $n$(=$h$'s length).) If, $$ Ah=0 $$ then $h$ can be the last column of $V$ where $A = UDV^T $. ($UDV^T$...
0
votes
1answer
12 views

Should the correlation PCA projection be computed on original or normalized samples?

Suppose we compute the correlation PCA of a dataset $X$ (with $m$ variables and $n$ observations) by first normalizing the input variables. That is: mean $\rightarrow 0$ and standard deviation $\...
0
votes
1answer
23 views

Nearest singular matrix

Let the SVD of $A \in \mathbb R^{{n}*{n}} $ be given as $A=\sum_{i=0}^n \sigma_{i}u_{i}v_{i}^{T}$ where $\sigma_{1}\gt \sigma_{2}>{...}>\sigma_{n-1}=\sigma_{n}>0 $ Compute a matrix $B$ such ...
1
vote
0answers
19 views

Finding a maximal orthogonal basis from a set of functions

I have a set of functions $\{f_1,\ldots,f_n\}$ with an associated inner product $\langle f_j,f_k\rangle=\int d^2z f_1^*f_2$ . The functions are not linearly dependent; i.e. the rank $r<n$, where $...
0
votes
0answers
12 views

Compute a similarity matrix of documents from SVD (LSI)

I'm computing a SVD from a Matrix $m$ (columns = Documents ($D$) and rows = Terms ($T$) and truncate this matrix to lower the dimension of $m$ to $k$. From my resulting matrix $A = U \Sigma V^T$ I ...
3
votes
0answers
27 views

SVD - Decomposed Matrix Sizes

I had a question about SVD. Specifically about the size of matrices $U$, $\Sigma$ and $V$ decomposed from the $m\times n$ matrix $X$ using the formula $$X = U \Sigma V^T $$ Most of the the tutorial ...
0
votes
0answers
14 views

Singular value decomposition of the real part of a complex matrix

An $N\times N$ complex matrix $K$, with SVD of $K = U\Sigma V^H $. I only have access to the real part of the elements of $K$, i.e. $K_R$. If I did SVD of $K_R$ to get $U_R \Sigma_R V_R^H$, my ...
2
votes
0answers
30 views

characterization of the solution to a generalized eigenvalue problem

Let's say we have the following optimization problem. (All the $\Sigma_{ii}$'s are positive definite.) $\max u^\top \Sigma_{12} v\quad$ $\text{subject to}\quad u^\top \Sigma_{11} u = 1\quad and\quad ...
2
votes
1answer
44 views

Solving $Ax = 0$ with Singular Value Decomposition

In my computer vision class, we have seen the problem of solving $Ax = 0$ numerous times. Initially, we solved the problem by computing the smallest eigenvectors corresponding to the smallest ...
2
votes
1answer
60 views

Matrix norm inequality : $\| Ax\| \leq |\lambda| \|x\|$, proof verification

Suppose that $A$ is a normal $n\times n$ matrix. Show that $\|Ax \| \geq |\lambda_n|\|x\|$ for all $x \in \mathbf{C}^n$, if $\lambda_n$ is the eigenvalue to $A$ of smallest absolute value. Is this ...
9
votes
2answers
125 views

Most efficient method for computing Singular Value Decomposition of a triangular matrix?

There are several methods available for computing SVD of a general matrix. I am interested in knowing about the best approach which could be used for computing SVD of an upper triangular matrix. ...
1
vote
1answer
57 views

Do similar matrices have equal singular values?

Is it true that if $A$ and $B$ are similar matrices, $B=S^{-1}AS$, then $A$ and $B$ have the same singular values?
0
votes
0answers
22 views

Singular Value Decomposition of $A=W^T V W$

I've given the following Matrix $A=W^{\dagger} V W$ where $\dagger$ denotes transpose+complex conjugation. It follows that $A$ is hermitian. $W$ is a $n \times m$ Matrix. $V$ is a diagonal $n\times ...
0
votes
0answers
10 views

Derivative of the singular values of the product of two matrices.

If $\mathbf{\Sigma}_{\mathbf{A}^H \mathbf{B}}$ is the diagonal matrix of singular values of $\mathbf{A}^H \mathbf{B}$, what is the derivative of $u_j=\operatorname{Tr}\left(\mathbf{\Sigma}_{\mathbf{A}^...
0
votes
0answers
12 views

SVD of a product of a Hermitian matrix and diagonals matrices

I have a Hermitian matrix $H$ and its SVD decomposition $H = V*S*V^T$. Let $D$ be a diagonal matrix. Is it possible to deduce the SVD of $D*H*D$ from the above SVD ?
2
votes
3answers
99 views

Understanding a derivation of the SVD

Here's an attempt to motivate the SVD. Let $A \in \mathbb R^{m \times n}$. It's natural to ask, in what direction does $A$ have the most "impact". In other words, for which unit vector $v$ is $\| A ...
1
vote
1answer
28 views

SVD of a real symmetric matrix confusion.

For a real symmetric matrix A, it's true that: $ A A^T = A^T A = A^2$ And since the right and left singular vectors of $A$ are the eigenvectors of $A^T A$ and $A A^T$ respectively, the right and ...
1
vote
0answers
25 views

Relationship between eigenvectors of correlation and covariance matrices

For the purpose of computing principal components of a dataset, represented as matrix $X$ of dimensions $n \times p$ with $n$ samples and $p$ features, we can compute sample covariance matrix $S$, and ...
1
vote
1answer
72 views

why do we say SVD can handle singular matrx when doing least square? Comparison of SVD and QR decomposition

I don't quite understand why we say that QR decomposition doesn't handle singular matrix, while SVD does when they are used for least square problem? My example in Matlab seems to support the ...