In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix, with many useful applications in signal processing and statistics.

learn more… | top users | synonyms

0
votes
2answers
18 views

Creating a random square matrix with known singular values

The first step in one question has me creating a random square matrix A with singular values given as $2^{-1}, 2^{-2}\dots 2^{-n}$. There is no other information about what assumptions can be made ...
0
votes
1answer
13 views

How to prove $V*V^T=I$ in SVD? [duplicate]

How to prove $V*V^T=I$ in SVD: $M=U*S*V^T$? It's easy to understand $V^T*V=I$. It seems $V*V^T=I$, but how to prove it?
0
votes
0answers
5 views

Matrix with highly correlated adjacency entries

I am learning about SVD from this book. One of the exercise questions asks me to create matrix with highly correlated adjacency entries and then conduct some experiments to discover the nature of the ...
0
votes
1answer
42 views

Why does SVD provide the least squares solution to $Ax=b$?

I am studying the Singular Value Decomposition and its properties. It is widely used in order to solve equations of the form $Ax=b$. I have seen the following: When we have the equation system $Ax=b$, ...
0
votes
0answers
14 views

Prove this relation between truncated SVD and eigen decomposition?

For a real matrix $M$, we have a full SVD $M=USV^T$ and a truncated SVD $M_{k}=U_kS_kV_k^T$. The truncated SVD means (matlab grammar): $U_k=U(:,1:k), S_k=S(1:k,1:k), V_k=V(:,1:k)$. Based on the ...
0
votes
0answers
22 views

Perpendicular distances versus vertical distances

Why is it better to use perpendicular distance rather than vertical distance along a particular coordinate axis when finding the best fit subspace? This is an exercise question in a chapter related to ...
1
vote
0answers
11 views

Bound on Signal Amplitude for subspace methods (MUSIC, ESPRIT)

MUSIC and ESPRIT are methods that use subspace decomposition to identify signal Parameters. Subspace decomposition is achieved either by SVD or Eigen Value Decomposition. Subspace decomposition ...
2
votes
1answer
42 views

How do I find 2x2 orthonormal diagonalizing matrices using only trigonometry?

I have a matrix $A=\begin{bmatrix} a & b \\ c & d \end{bmatrix}$ (where all values are known), and I eventually want to diagonalize it into: $$ A=UDV^T $$ for orthonormal U and V. If I ...
0
votes
0answers
14 views

Issues with connecting the SVD and Eigenvalues for block matrix

In class, we have talked about the singular value decomposition and its connection to Eigenvalues. Specifically, for a matrix A, if the columns of a matrix contain linearly independent eigenvectors, ...
0
votes
1answer
40 views

Plane fitting using svd

I am trying to get a best fit plane in a 3d space of points. I am using an svd as described in http://stackoverflow.com/questions/10900141/fast-plane-fitting-to-many-points. If I use the data provided ...
1
vote
0answers
24 views

Double Normalization Before SVD

It is usually suggested that we normalize a matrix before SVD. For a certain recommender application we found out normalizing both columns and rows is necessary otherwise SVD based recommendations do ...
0
votes
0answers
27 views

Modifying U=mxn SVD Algorithm to U=mxm Algorithm

I have painstakingly ported this Python source "svd.py" to C++. I confirm it works for the example it comes with. While testing another example (this one, from Wikipedia), the assert statement trips ...
1
vote
0answers
9 views

What is pseudodiagonality in matrix/tensor?

What is the difference between diagonality and pseudodiagonality? Does this apply to tensor too? https://www.math.uzh.ch/fileadmin/math/preprints/06_11.pdf
0
votes
1answer
40 views

What is the Singular Value Decomposition for the Zero Matrix?

I am interested in the singular value decomposition of a matrix: $\mathbf{M} = \mathbf{U} \mathbf{S} \mathbf{V}^T$. Suppose $\mathbf{M} = \mathbf{0}$ (zero matrix) and square. Clearly, $\mathbf{S} = ...
5
votes
1answer
105 views

Can I turn $Ax=b$ into $Ax=0$?

For a system of equations $$ \begin{bmatrix}d_1 & d_2 & \dots & d_n \end{bmatrix} \begin{bmatrix}u_1\\u_2\\ \vdots \\ u_n \end{bmatrix} = d_{n+1} $$ where each $d$ is a column of ...
0
votes
1answer
18 views

Idea of singular vectors in matrix completion

I am trying to understand the concept of matrix completion. I came across the following line (M is a low rank matrix, and we only have few samples of it. The task is to recover the entire matrix from ...
1
vote
1answer
75 views

Do these two rearranged matrices have the same singular values (or the same rank)?

This is the origin of my problem: I have a set of data which expresses which user ($U$ set) applies what tag ($T$ set) to which item ($I$ set). So it is actually a $U×I×T$ tensor $A$ (or 3-dimensional ...
0
votes
1answer
26 views

How to find parameters that minimize the sum of squares, using Matlab?

I have a system of linear equations in the following form. How can I solve it in Matlab? $$\operatorname*{argmin}_{a,b} \sum_{i,j} [X(i,j)-a\times Y(i,j)-b]^2$$ Where X and Y are known. I need to ...
0
votes
0answers
19 views

Weighted Rigid Body Transformation

Usually if one talks about rigid body transformation between 2 sets of points, it means: Performing rigid body transformation upon 1 set of points so that the least square error between the 2 sets of ...
0
votes
0answers
33 views

Is the eigenvalue decomposition equal to the singular value decomposition for real symmetric matrices?

Question is as the title states. I've read something similar for hermitian matrices, but am unsure if this is correct as well for real symmetric matrices.
0
votes
1answer
28 views

Finding SVD of a Matrix

If $a_{1},a_{2} \in \mathcal{R}^{2},$ $\ \|a_{1}\|_{2} = \|a_{2}\|_{2} = K$, and the angle $\theta$ between $a_{1}$ and $a_{2}$ is between $0$ and $\pi/2$, we want to compute the SVD of the matrix $A ...
1
vote
1answer
42 views

Find rigid transform from coordinates of the same points in different reference frames

Given two* 3-dimensional points $p_1$ and $p_2$ expressed in different reference frames $A$ and $B$, find the rigid transform (rotation and translation) between frames $A$ and $B$. The answer to this ...
1
vote
1answer
19 views

Cross-product is a left singular vector?

Assume A is a 3x2 matrix with rank(A)=2. u1 and u2 are already left singular vectors... How would I go about proving that the cross-product of the two is also a left singular vector? Hints would be ...
1
vote
1answer
83 views

Efficient Algorithm for Iteratively Reweighted Least Squares Problem

I'm interested in solving a weighted least squares problem of the form $X^T W X \beta = X^T W Y$ where $W$ is a diagonal, positive definite matrix, $X \in R^{m \times n}$, $Y \in R^{m \times 1}$ and ...
1
vote
1answer
36 views

Why diagonal matrix SVD sorted from largest to smallest value?

Why diagonal matrix SVD sorted from largest to smallest value? D is diagonal matrix, $D=(d_1 \ge ,d_2 \ge ,..., \ge d_L)$. Whether there is a journal that could explain this?
0
votes
0answers
24 views

PCA eigenvalues

When projecting the data set on the Eigen vectors of the co-variance matrix , the eigenvalues represent how much each example varies away from the mean of the data set in the projected direction , ...
0
votes
0answers
23 views

Inverse Square Root Diagonal Matrix

In QUEST algorithm, there is step to "transform a categorical predictor into a continuous predictor" : Let $X$ be a nominal categorical predictor taking values in the set $(b_1,...,b_L)$. Transform ...
0
votes
0answers
50 views

How to explain this matrix?

In QUEST algorithm, there is step to "transform a categorical predictor into a continuous predictor" : Let $X$ be a nominal categorical predictor taking values in the set $(b_1,...,b_L)$. Transform ...
1
vote
0answers
30 views

Change in Singular Value Decomposition of a matrix on addition of a single row

Given that I know the svd decomposition of a matrix, is there any way to compute the svd decomposition of the matrix obtained by adding a single row to the original matrix? Is there any relation ...
0
votes
0answers
37 views

Proof of Singular Value Decomposition

I found an interesting property of SVD in the book of Introduction to Information Retrieval by Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, page 408. The question is can I use the ...
0
votes
1answer
25 views

how to do SVD using the covariance matrix

I have an $(N \times M)$ matrix $A$ with $M \gg N$, $M$ being millions and $N$ hundreds, and I want to do $SVD$ on the matrix $A$. Can I do this calculation using $A\cdot A$ (the covariance matrix)?
1
vote
0answers
41 views

Finding the knee point in an eigenvalue plot

I want to automatically find the "knee" point of the eigenvalue plot (or also called elbow of the scree plot). I.e. I have a vector of eigenvalues (sorted from highest to lowest) and I want some ...
3
votes
2answers
176 views

Relationship between the singular value decomposition (SVD) and the principal component analysis (PCA). A radical result(?)

I was wondering if I could get a mathematical description of the relationship between the singular value decomposition (SVD) and the principal component analysis (PCA). To be more specific I have ...
3
votes
0answers
50 views

Moore-Penrose Pseudo-inverse of a matrix on adding 1 new row/column

Given that I know the pseudo-inverse of a matrix(not necessarily a square matrix), how to calculate the pseudo-inverse of the matrix I get by adding a single row/column to the original matrix? i.e, ...
0
votes
0answers
21 views

find indexes for tensorial approximation

let us consider following matrix where $K$ and $L$ are dimensions and and $L=N-K+1$,where $N$ is length of given vector,namely $x=(x_1,x_2,x_3,....x_k,x_{k+1},x_{k+2},...x_N)$ please pay ...
1
vote
1answer
37 views

represent hankel matrix by low rank tensorial approximation

suppose that we have given following matrix \begin{matrix} x_1 & x_2 & ..x_p \\ x_2 & x_3 & ...x_{p+1} \\ . & .& . & \\ x_{N-p+1} & x_{n-p+2} &... x_n ...
2
votes
1answer
24 views

Spectral norm of a matrix obtained by setting some entries to zero

For example can we say, that if $A$ is original matrix and $A'$ obtained from $A$ by zeroing some elements then $\|A\|_2 \geq \|A'\|_2$?
3
votes
2answers
49 views

Singular Value Decomposition of the pseudo-inverse of a matrix

There is $A$ which is a matrix: $$\begin{bmatrix}2 & 4 \\ 1 & -4 \\ -2 & 2\end{bmatrix}.$$ While I have easily worked out the singular value decomposition of this matrix, but I am not ...
0
votes
1answer
38 views

Eigenvalue decomposition of $2\times 2$ matrix

There is a matrix $$A = \begin{bmatrix}3 & 1\\ 1 & 3 \end{bmatrix}$$ and the factorisation of $A = S*D*S^T$ needs to figured out. So far I have figured out that the eigensolutions are 4 and 2 ...
0
votes
0answers
16 views

Total Least Squares problem when some columns of data matrix have no error

I'm reading through Golub and Van Loan and they mention that to solve the total least-squares problem $(A + E)x = b + r$, where the first $s$ columns of E are zero, then we can solve the problem by ...
1
vote
0answers
30 views

Do signs matter in SVD?

I have written an algorithm to compute the SVD of a 2x2 matrix. I was checking against a Mathematica query, and I noticed that the signs in the $U$ and $V$ matrices do not match those from my ...
1
vote
1answer
29 views

Why is the first left and right singular vectos scale by the first singular values a good approximation of the original matrix

Conceptually, why is the first singular vector a good rank one approximation instead of something like the averaging of the total singular vectors? If you have $$A = U\Sigma V^T $$ why isn't ...
0
votes
0answers
24 views

Is there a way to control the variance of the singular values in SVD?

I have an engineering problem in which I use SVD on matrix $\mathbf{A}$: \begin{align} \mathbf{A} &= \textbf{U} \mathbf{\Sigma} \textbf{V}^{*} \end{align} However, due to the fact that the ...
1
vote
1answer
24 views

How is it possible to solve for singular values of a matrix and how is it different than solving for eigen values?

I am in the process of teaching myself about singular values, SVD and eigenvects.. etc. I am looking at a question asking to find the singular values of a $2\times 3$ matrix, but am unsure what this ...
0
votes
0answers
40 views

Some questions about svd proofs and linear algebra

Theorem: The rank of A is r, the number of nonzero singular values. Proof: The rank of a diagonal matrix is equal to the number of its nonzero entries, and in the decomposition $A=U\Sigma{}V^*$, U ...
1
vote
1answer
36 views

Proof concerning eigen values

Could somebody help me into proving this theorem? if $A$ and $B^{H}$ are in $C^{m\times n}$ with $m\geq n$, then $\lambda (AB) = \lambda(BA) \cup \lbrace 0, \ldots ,0\rbrace.$ Thenks, Elnaz
1
vote
1answer
94 views

Multicollinearity and SVD

I compute the Singular Value Decomposition of a n x n matrix. If the matrix is not full rank, and I have 2 collinear columns, I end up with one singular value equal to 0. Is it possible to find out ...
2
votes
1answer
356 views

Proof of Eckart-Young-Mirsky theorem

Could someone please explain why in http://en.wikipedia.org/wiki/Low-rank_approximation#Proof_of_Eckart.E2.80.93Young.E2.80.93Mirsky_theorem it says "we know that $\exists(k+1)$ dimension space ...
2
votes
1answer
52 views

SVD: proof of existence

I'm reading "Numerical Linear Algebra" by Lloyd Thefethen. For Singular Value Decomposition proof of existence it starts like this: "Set $\sigma_1=||A||_2$. By a compactness argument, there must be ...
2
votes
2answers
45 views

Uniqueness of Singular Values

Given a matrix A, one inductively constructs (and thus proving its very existence) the singular value decomposition as follows: take $ \sigma_{1}=||A||_{2} $, and consider a couple of vectors such ...