In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix, with many useful applications in signal processing and statistics.

learn more… | top users | synonyms

0
votes
0answers
11 views

Best “right approximation” to a matrix?

I know the best low-rank approximation to a matrix in the least-squares sense is given by the truncated SVD, but I'm trying to do something a little different. Given an X that's MxN I want to find ...
1
vote
1answer
21 views

Multicollinearity and SVD

I compute the Singular Value Decomposition of a n x n matrix. If the matrix is not full rank, and I have 2 collinear columns, I end up with one singular value equal to 0. Is it possible to find out ...
1
vote
1answer
33 views

Proof of Eckart-Young-Mirsky theorem

Could someone please explain why in http://en.wikipedia.org/wiki/Low-rank_approximation#Proof_of_Eckart.E2.80.93Young.E2.80.93Mirsky_theorem it says "we know that $\exists(k+1)$ dimension space ...
1
vote
1answer
21 views

SVD: proof of existence

I'm reading "Numerical Linear Algebra" by Lloyd Thefethen. For Singular Value Decomposition proof of existence it starts like this: "Set $\sigma_1=||A||_2$. By a compactness argument, there must be ...
2
votes
2answers
31 views

Uniqueness of Singular Values

Given a matrix A, one inductively constructs (and thus proving its very existence) the singular value decomposition as follows: take $ \sigma_{1}=||A||_{2} $, and consider a couple of vectors such ...
1
vote
0answers
7 views

recovering of time series in SSA

i am trying to reconstruct time series from SSA ,because according to this link http://en.wikipedia.org/wiki/Singular_spectrum_analysis there is procedure ...
2
votes
0answers
48 views

How to FAST calculate 2 norm / spectral norm of a matrix.

I meant reduced 2 norm, the largest singular value. My current approach is applying the SVD decomposition of A via "?gesdd" in MKL, and then taking the largest singular value. I think there should ...
4
votes
1answer
32 views

Maximizing the trace

Say i have the following maximization. $ max_R$ trace $(RZ): R^TR = I_n$ where $R$ is an $n$ x $n$ orthogonal transformational vector. Also, the SVD of $Z = USV^T$. I'm trying to find the optimal ...
3
votes
1answer
40 views

Is there a faster way to calculate a pseudo-inverse of a matrix than using SVD that is as numerically stable as with SVD?

Is there a faster way to calculate a pseudo-inverse of a matrix than using SVD that is as numerically stable as using SVD?
5
votes
4answers
298 views

Mysterious Proof about Induced Norms (was: Uniqueness of SVD)

In order to prove non-uniqueness of singular vectors when a repeated singular value is present, the book (Trefethen), argues as follows: Let $\sigma$ be the first singular value of A, and $v_{1}$ the ...
0
votes
2answers
38 views

Linear Equation system - solve only solvable variables

If I have the SVD of a Matrix $A$, how do I solve the linear equation system $Ax=b$? The problem is that if I e.g. has this linear equation system: $-2y + z = 3$ $-4y + 2z = 6$ $x -2y + z = 4$ ...
0
votes
2answers
38 views

About matrix $R$, what is this called: $R^TR$? What is it for?

I am doing singular value decomposition on a matrix $R$. The first step is to compute such a matrix $R^TR$. What is this matrix? A reference told me this is cross product of matrix R. I use a ...
2
votes
1answer
58 views

Eigenvalues of $ADA^T$

Consider a rectangular matrix $A\in\mathbb{R}^{M\times N}$ and a diagonal matrix $D\in\mathbb{R}^{N\times N}$. What can one say on the eigenvalues and eigenvectors of $ADA^T$? For example, if we ...
1
vote
0answers
37 views

Using SVD in PCA for image compression

I found some help material and guided by it tried to implement PCA using SVD in MAtlab for image compression. I did it in this way: ...
1
vote
0answers
17 views

Some terminology and reference questions on singular values

Let $T: V \rightarrow W$ be an operator between to inner product spaces. Then singular values $s_1 \leq s_2 .... \leq s_n$ of $T$ are square roots of eigenvalues of $T^*T$ where $T^*$ is the conjugate ...
0
votes
0answers
12 views

Question regarding a non-standard formulation of the SVD Theorem

In a course I'm taking the following theorem was presented as "The SVD Theorem": Theorem: Let $A\in\mathbb{R}^{d\times m}$ be a matrix of rank $r\leq\min\left\{ m,d\right\}$ . Then $A$ admit a ...
0
votes
0answers
30 views

Canonical Correlation Analysis (CCA) by SVD

We all know that CCA correlation vectors can be computed by eigen-decomposition (using function eig() in MATLAB) as shown here. For example: eig($C_{xx}^{-1}C_{xy}C_{yy}^{-1}C_{yx}$) (1) Recently I ...
1
vote
0answers
45 views

Linear systems and bilinear forms

Given a general linear system $Ax = y$ and the bilinear form $z(x,y) = y^T Ax$, what are the links between these two mathematical objects? Thanks. EDIT: Original question is too general and ...
0
votes
3answers
52 views

About matrix products $A^{T}A$ and $ AA^{T} $

I'm investigating the relationship between 2-norms and eigenvalues of $A^{T}A$ and $ AA^{T} $, in order to better understand the SVD decomposition. How can I prove that $A^{T}A$ and $ AA^{T} $ are ...
1
vote
1answer
21 views

Largest Singular value of a Matrix

Prove that if $A\in \mathbb{R}^{m\times n}$, then $$\sigma_{\text{max}} (A) = \underset{y\in\mathbb{R}^m\\x\in \mathbb{R}^n}{\text{max}}\frac{y^TAx}{\Vert{x}\Vert_2\Vert y\Vert_2}.$$
0
votes
1answer
33 views

Inverse Square root of a rectangular matrix

I am trying to compute the inverse square root ($X^{-1/2}$) of a $n \times p$ matrix with $n > p$. I was wondering if we can compute it via SVD just as we do it for square diagonalizable matrices ...
1
vote
1answer
30 views

SVD : How is the normalized statistical leverage score a probability distribution?

I came across a paper related to the Singular Value Decomposition (SVD) where they calculate a normalized statistical leverage score for each column of a $m \times n$ matrix, It is defined as follows: ...
0
votes
1answer
35 views

Singular Value Decomposition of Rank 1 matrix

I am trying to understand singular value decomposition. I get the general definition and how to solve for the singular values of form the SVD of a given matrix however, I came across the following ...
0
votes
1answer
40 views

calculate svd - example with roots

Do an Singular Value Decomposition of $$ \begin{bmatrix} 0 & \sqrt{2} & 0 & \sqrt{2} \\ \sqrt{2} & 0 & \sqrt{2} & 0 \end{bmatrix}$$ I have tried to find it following the ...
2
votes
1answer
26 views

Relationship betwen image components and SVD?

Let's say I have an image representing a sampled function. It just so happens that I know this function can be represented as a sum of individual outer products along with some noise. So I might ...
0
votes
0answers
33 views

Equivalent of SVD for 3D motion capture time series

I work in image science but would like to analyse some motion capture time series data. If it were an image time series, where each point has a scalar value, I would run an SVD to explore the ...
0
votes
1answer
24 views

The Geometry of a Linear Transformation

Consider a square matrix of full rank (these assumptions are made for the sake of simplicity). This matrix expresses in coordinates a Linear Mapping that sends the unit sphere to a hyperellipsoid on ...
2
votes
1answer
48 views

Parallel Algorithms for SVD

I just have completed a preliminar theoretical study of the important SVD decomposition. Now, I'm moving to numerical calculation of SVD. I would like to learn directly a parallel algorithm to ...
0
votes
1answer
82 views

PCA using SVD in Matlab, a few questions.

I have X = [25, 2000] i.e. 25 subjects and 2000 values (i.e. each subject has a spectrogram that is reduced to 2000 values). My goal is to reduce from 25 subjects to 1 or 2 "subjects" that best ...
0
votes
0answers
12 views

Solve a overdetermined linear system

I have a linear system with 4 variables $x=(x_1, x_2, x_3, x_4)$ and 6 equations, $Mx=b$. The system is overdetermined, but the rank of the matrix $M$ of the coefficients is 4. This means that a ...
1
vote
0answers
17 views

When QR outperform SVD to estimate column space of a symmetric matrix?

I have a symmetric matrix, $\hat{A}, $which is an estimator of another true symmetric matrix, $A$. I would like to estimate the column space of $A$. I know I can do QR-Algorithm or use spectral ...
0
votes
2answers
46 views

Can the singular values be plotted?

I have a set of 3 x 3 co-variance matrices that I need to plot. Using singular value decomposition (svd) in Matlab I managed to obtains a vector of singular values for each matrix as stated here. I ...
1
vote
1answer
41 views

SVD for a matrix with a given orthonormal $\mathbf{U}$

Let: $\mathbf{A}$ be a $N\times N$ complex matrix. $\mathbf{u}\in \operatorname{span}(\mathbf{A})$ be a given unit norm vector, where $\operatorname{span}(\mathbf{A})$ denotes the column space of ...
0
votes
0answers
56 views

low-rank matrix approximation (fast partial SVD)

I have a question on low-rank matrix approximation. Find a matrix $X^{\star}$ that is the solution to the following problem: $$ X^{\star} =\min_{rank(X)\le r} \|A-X\|_F^2 $$ I know that this problem ...
1
vote
0answers
37 views

How does SVD work?

Trying to find information, and, no-one seems to know the answers. I have a time-series, represented by $T = [0, 1, 1, 0, \ldots, n]$ the time series is then transformed into the Spectral results: ...
1
vote
1answer
21 views

Differing definitions of Matrix Condition Number

When describing the condition number of a correlation matrix I have seen is described as the ratio of the singular values (from singluar value decomposition). I have also seen it described as the ...
1
vote
2answers
73 views

union of two linear independent set

I have two linearly independent sets in $R^n$ space.One with cardinality $n-k$ and another with cardinality $k+1$.Is the union of these two sets linearly independent?Actually I came across this ...
3
votes
1answer
34 views

Specifics on using PCA via SVD … use data matrix or covariance matrix of data matrix

So I want to clarify something: if I want to use SVD to compute PCA of matrix X, do I need to use the SVD on $X$ or $X^TX$. If the former, do I need to square the eigenvalues returned by the SVD? If ...
1
vote
1answer
82 views

Time complexity of finding nullspace of a matrix

The problem is finding the nullspace of a singular $n \times n$ square matrix $A$ (or alternatively computing the eigenvectors corresponding to the eigenvalue 0). What is the algorithm with the ...
1
vote
1answer
45 views

Understanding (and computing) the svd of a matrix

I thought I understood how to compute the singular value decomposition $A = U D V^T$ of a matrix $A$, based on this tutorial: ...
3
votes
1answer
37 views

Calculate the pseudo inverse of the matrix

The subject is to calculate the pseudo inverse if matrix $\begin{equation*} \mathbf{A} = \left( \begin{array}{ccc} 1 & 0 \\ 2 & 1 \\ 0 & 1 \\ \end{array} ...
1
vote
1answer
43 views

Singular Value Decomposition (SVD). Show equality.

I have to show the following equality: $WW^{T} = USUS$, where $W = USV^{T}$; $U,V$ are orthogonal matrices and $S$ is a diagonal matrix, where all entries but the diagonal are $0$. I think I am ...
1
vote
0answers
51 views

SVD maps from orthogonal basis of rowspace to orthogonal basis in column space?

I've been going through the MIT OpenCourseWare Linear Algebra course, and I've been understanding it until early in the Singular Value Decomposition video. The Professor says (and I'm paraphrasing) ...
1
vote
2answers
30 views

Can you compute rank r factorization of a n*n matrix in time O(n^2 r)?

I am wondering if you can compute the SVD/eigenvectors of a rank r matrix of size n*n in time O(n^2 r)? My understanding is that standard eigenvector computations involve bringing matrix into ...
0
votes
0answers
44 views

How to multiply the decomposed matrices from a truncated SVD

I'm reading this: http://nlp.stanford.edu/IR-book/pdf/18lsi.pdf I'm trying to understand how you reduce the number of dimensions in a matrix. There's an example on page 13 which I'm trying to ...
2
votes
1answer
66 views

SVD. why U and V have to be both orthonormal matrices?

I'm looking for the SVD factorization $A = U D V'$ starting from the set of equations $A u = v d$ and $A' v = u d$. Where u and v are vectors from the A and A' spaces and d the singular value. ...
3
votes
1answer
75 views

Understanding higher order SVD

Can someone explain the singular value decomposition of a tensor (maybe a 3 dimensional matrix) with an example? It is intuitively difficult to the get the meaning from just the formulas. On a ...
0
votes
0answers
23 views

simple question on SVD

I have an algorithm that outputs a vector 'v' for every iteration. With ever iteration the change in vector v will get smaller and smaller. therefore my psuedo code should be if vold-v < 0.001 ...
3
votes
1answer
29 views

Estimate loss of information due to a low rank approximization by SVD

I have a matrix $X$ and I compute its Singular Value Decomposition: $$X = U \Sigma V^T$$ then, I take the lower rank approximization: $$X_k = U_k \Sigma_k V^T_k$$ where $k < rank(X)$, $U_k$ is made ...
0
votes
0answers
20 views

Calculating eigenvlaues of X'X using svd

Normally when one calculates the singular values for X these are the square roots of the matrix X'X. However here the X variables have been normalised so that $(x_{i,j}-\hat{x_j})/\sqrt{n-1}*s$ ...