In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix, with many useful applications in signal processing and statistics.

learn more… | top users | synonyms

0
votes
1answer
19 views

Finding SVD of a Matrix

If $a_{1},a_{2} \in \mathcal{R}^{2},$ $\ \|a_{1}\|_{2} = \|a_{2}\|_{2} = K$, and the angle $\theta$ between $a_{1}$ and $a_{2}$ is between $0$ and $\pi/2$, we want to compute the SVD of the matrix $A ...
1
vote
1answer
24 views

Find rigid transform from coordinates of the same points in different reference frames

Given two* 3-dimensional points $p_1$ and $p_2$ expressed in different reference frames $A$ and $B$, find the rigid transform (rotation and translation) between frames $A$ and $B$. The answer to this ...
1
vote
1answer
16 views

Cross-product is a left singular vector?

Assume A is a 3x2 matrix with rank(A)=2. u1 and u2 are already left singular vectors... How would I go about proving that the cross-product of the two is also a left singular vector? Hints would be ...
1
vote
1answer
22 views

Efficient Algorithm for Iteratively Reweighted Least Squares Problem

I'm interested in solving a weighted least squares problem of the form $X^T W X \beta = X^T W Y$ where $W$ is a diagonal, positive definite matrix, $X \in R^{m \times n}$, $Y \in R^{m \times 1}$ and ...
0
votes
0answers
13 views

What is the relationship between SVD and Spectral Decomposition in Matrix symmetric?

What is the relationship between SVD and Spectral Decomposition in Matrix symmetric? SVD A=U$\Sigma$$V^T$ and Spectral Decomposition A=PD$P^T$, If both have orthogonal and othonormal matrix?
1
vote
1answer
30 views

Why diagonal matrix SVD sorted from largest to smallest value?

Why diagonal matrix SVD sorted from largest to smallest value? D is diagonal matrix, $D=(d_1 \ge ,d_2 \ge ,..., \ge d_L)$. Whether there is a journal that could explain this?
0
votes
0answers
18 views

PCA eigenvalues

When projecting the data set on the Eigen vectors of the co-variance matrix , the eigenvalues represent how much each example varies away from the mean of the data set in the projected direction , ...
-1
votes
0answers
14 views

pca data recovery equations

In PCA , consider data matrix 4 x 3 ( 4 examples each with 3 features ) , after getting the 3 eigen vectors (a/b/c) and projecting data on the first 2 vectors , the eqn looks like this : [ first 2 ...
0
votes
0answers
20 views

Inverse Square Root Diagonal Matrix

In QUEST algorithm, there is step to "transform a categorical predictor into a continuous predictor" : Let $X$ be a nominal categorical predictor taking values in the set $(b_1,...,b_L)$. Transform ...
0
votes
0answers
49 views

How to explain this matrix?

In QUEST algorithm, there is step to "transform a categorical predictor into a continuous predictor" : Let $X$ be a nominal categorical predictor taking values in the set $(b_1,...,b_L)$. Transform ...
1
vote
0answers
18 views

Change in Singular Value Decomposition of a matrix on addition of a single row

Given that I know the svd decomposition of a matrix, is there any way to compute the svd decomposition of the matrix obtained by adding a single row to the original matrix? Is there any relation ...
0
votes
0answers
23 views

Proof of Singular Value Decomposition

I found an interesting property of SVD in the book of Introduction to Information Retrieval by Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, page 408. The question is can I use the ...
0
votes
1answer
22 views

how to do SVD using the covariance matrix

I have an $(N \times M)$ matrix $A$ with $M \gg N$, $M$ being millions and $N$ hundreds, and I want to do $SVD$ on the matrix $A$. Can I do this calculation using $A\cdot A$ (the covariance matrix)?
1
vote
0answers
28 views

Finding the knee point in an eigenvalue plot

I want to automatically find the "knee" point of the eigenvalue plot (or also called elbow of the scree plot). I.e. I have a vector of eigenvalues (sorted from highest to lowest) and I want some ...
3
votes
2answers
122 views

Relationship between the singular value decomposition (SVD) and the principal component analysis (PCA). A radical result(?)

I was wondering if I could get a mathematical description of the relationship between the singular value decomposition (SVD) and the principal component analysis (PCA). To be more specific I have ...
3
votes
0answers
42 views

Moore-Penrose Pseudo-inverse of a matrix on adding 1 new row/column

Given that I know the pseudo-inverse of a matrix(not necessarily a square matrix), how to calculate the pseudo-inverse of the matrix I get by adding a single row/column to the original matrix? i.e, ...
0
votes
0answers
21 views

find indexes for tensorial approximation

let us consider following matrix where $K$ and $L$ are dimensions and and $L=N-K+1$,where $N$ is length of given vector,namely $x=(x_1,x_2,x_3,....x_k,x_{k+1},x_{k+2},...x_N)$ please pay ...
1
vote
1answer
30 views

represent hankel matrix by low rank tensorial approximation

suppose that we have given following matrix \begin{matrix} x_1 & x_2 & ..x_p \\ x_2 & x_3 & ...x_{p+1} \\ . & .& . & \\ x_{N-p+1} & x_{n-p+2} &... x_n ...
2
votes
1answer
22 views

Spectral norm of a matrix obtained by setting some entries to zero

For example can we say, that if $A$ is original matrix and $A'$ obtained from $A$ by zeroing some elements then $\|A\|_2 \geq \|A'\|_2$?
3
votes
2answers
36 views

Singular Value Decomposition of the pseudo-inverse of a matrix

There is $A$ which is a matrix: $$\begin{bmatrix}2 & 4 \\ 1 & -4 \\ -2 & 2\end{bmatrix}.$$ While I have easily worked out the singular value decomposition of this matrix, but I am not ...
0
votes
1answer
36 views

Eigenvalue decomposition of $2\times 2$ matrix

There is a matrix $$A = \begin{bmatrix}3 & 1\\ 1 & 3 \end{bmatrix}$$ and the factorisation of $A = S*D*S^T$ needs to figured out. So far I have figured out that the eigensolutions are 4 and 2 ...
0
votes
0answers
14 views

Total Least Squares problem when some columns of data matrix have no error

I'm reading through Golub and Van Loan and they mention that to solve the total least-squares problem $(A + E)x = b + r$, where the first $s$ columns of E are zero, then we can solve the problem by ...
1
vote
0answers
27 views

Do signs matter in SVD?

I have written an algorithm to compute the SVD of a 2x2 matrix. I was checking against a Mathematica query, and I noticed that the signs in the $U$ and $V$ matrices do not match those from my ...
1
vote
1answer
28 views

Why is the first left and right singular vectos scale by the first singular values a good approximation of the original matrix

Conceptually, why is the first singular vector a good rank one approximation instead of something like the averaging of the total singular vectors? If you have $$A = U\Sigma V^T $$ why isn't ...
0
votes
0answers
24 views

Is there a way to control the variance of the singular values in SVD?

I have an engineering problem in which I use SVD on matrix $\mathbf{A}$: \begin{align} \mathbf{A} &= \textbf{U} \mathbf{\Sigma} \textbf{V}^{*} \end{align} However, due to the fact that the ...
1
vote
1answer
23 views

How is it possible to solve for singular values of a matrix and how is it different than solving for eigen values?

I am in the process of teaching myself about singular values, SVD and eigenvects.. etc. I am looking at a question asking to find the singular values of a $2\times 3$ matrix, but am unsure what this ...
0
votes
0answers
35 views

Some questions about svd proofs and linear algebra

Theorem: The rank of A is r, the number of nonzero singular values. Proof: The rank of a diagonal matrix is equal to the number of its nonzero entries, and in the decomposition $A=U\Sigma{}V^*$, U ...
1
vote
1answer
34 views

Proof concerning eigen values

Could somebody help me into proving this theorem? if $A$ and $B^{H}$ are in $C^{m\times n}$ with $m\geq n$, then $\lambda (AB) = \lambda(BA) \cup \lbrace 0, \ldots ,0\rbrace.$ Thenks, Elnaz
1
vote
1answer
55 views

Multicollinearity and SVD

I compute the Singular Value Decomposition of a n x n matrix. If the matrix is not full rank, and I have 2 collinear columns, I end up with one singular value equal to 0. Is it possible to find out ...
2
votes
1answer
149 views

Proof of Eckart-Young-Mirsky theorem

Could someone please explain why in http://en.wikipedia.org/wiki/Low-rank_approximation#Proof_of_Eckart.E2.80.93Young.E2.80.93Mirsky_theorem it says "we know that $\exists(k+1)$ dimension space ...
1
vote
1answer
34 views

SVD: proof of existence

I'm reading "Numerical Linear Algebra" by Lloyd Thefethen. For Singular Value Decomposition proof of existence it starts like this: "Set $\sigma_1=||A||_2$. By a compactness argument, there must be ...
2
votes
2answers
35 views

Uniqueness of Singular Values

Given a matrix A, one inductively constructs (and thus proving its very existence) the singular value decomposition as follows: take $ \sigma_{1}=||A||_{2} $, and consider a couple of vectors such ...
1
vote
0answers
13 views

recovering of time series in SSA

i am trying to reconstruct time series from SSA ,because according to this link http://en.wikipedia.org/wiki/Singular_spectrum_analysis there is procedure ...
2
votes
1answer
71 views

How to FAST calculate 2 norm / spectral norm of a matrix.

I meant reduced 2 norm, the largest singular value. My current approach is applying the SVD decomposition of A via "?gesdd" in MKL, and then taking the largest singular value. I think there should ...
4
votes
1answer
48 views

Maximizing the trace

Say i have the following maximization. $ max_R$ trace $(RZ): R^TR = I_n$ where $R$ is an $n$ x $n$ orthogonal transformational vector. Also, the SVD of $Z = USV^T$. I'm trying to find the optimal ...
3
votes
1answer
53 views

Is there a faster way to calculate a pseudo-inverse of a matrix than using SVD that is as numerically stable as with SVD?

Is there a faster way to calculate a pseudo-inverse of a matrix than using SVD that is as numerically stable as using SVD?
5
votes
4answers
330 views

Mysterious Proof about Induced Norms (was: Uniqueness of SVD)

In order to prove non-uniqueness of singular vectors when a repeated singular value is present, the book (Trefethen), argues as follows: Let $\sigma$ be the first singular value of A, and $v_{1}$ the ...
0
votes
2answers
61 views

Linear Equation system - solve only solvable variables

If I have the SVD of a Matrix $A$, how do I solve the linear equation system $Ax=b$? The problem is that if I e.g. has this linear equation system: $-2y + z = 3$ $-4y + 2z = 6$ $x -2y + z = 4$ ...
0
votes
2answers
41 views

About matrix $R$, what is this called: $R^TR$? What is it for?

I am doing singular value decomposition on a matrix $R$. The first step is to compute such a matrix $R^TR$. What is this matrix? A reference told me this is cross product of matrix R. I use a ...
3
votes
1answer
76 views

Eigenvalues of $ADA^T$

Consider a rectangular matrix $A\in\mathbb{R}^{M\times N}$ and a diagonal matrix $D\in\mathbb{R}^{N\times N}$. What can one say on the eigenvalues and eigenvectors of $ADA^T$? For example, if we ...
1
vote
0answers
85 views

Using SVD in PCA for image compression

I found some help material and guided by it tried to implement PCA using SVD in MAtlab for image compression. I did it in this way: ...
1
vote
0answers
22 views

Some terminology and reference questions on singular values

Let $T: V \rightarrow W$ be an operator between to inner product spaces. Then singular values $s_1 \leq s_2 .... \leq s_n$ of $T$ are square roots of eigenvalues of $T^*T$ where $T^*$ is the conjugate ...
0
votes
0answers
14 views

Question regarding a non-standard formulation of the SVD Theorem

In a course I'm taking the following theorem was presented as "The SVD Theorem": Theorem: Let $A\in\mathbb{R}^{d\times m}$ be a matrix of rank $r\leq\min\left\{ m,d\right\}$ . Then $A$ admit a ...
0
votes
0answers
84 views

Canonical Correlation Analysis (CCA) by SVD

We all know that CCA correlation vectors can be computed by eigen-decomposition (using function eig() in MATLAB) as shown here. For example: eig($C_{xx}^{-1}C_{xy}C_{yy}^{-1}C_{yx}$) (1) Recently I ...
1
vote
0answers
46 views

Linear systems and bilinear forms

Given a general linear system $Ax = y$ and the bilinear form $z(x,y) = y^T Ax$, what are the links between these two mathematical objects? Thanks. EDIT: Original question is too general and ...
0
votes
3answers
59 views

About matrix products $A^{T}A$ and $ AA^{T} $

I'm investigating the relationship between 2-norms and eigenvalues of $A^{T}A$ and $ AA^{T} $, in order to better understand the SVD decomposition. How can I prove that $A^{T}A$ and $ AA^{T} $ are ...
1
vote
1answer
24 views

Largest Singular value of a Matrix

Prove that if $A\in \mathbb{R}^{m\times n}$, then $$\sigma_{\text{max}} (A) = \underset{y\in\mathbb{R}^m\\x\in \mathbb{R}^n}{\text{max}}\frac{y^TAx}{\Vert{x}\Vert_2\Vert y\Vert_2}.$$
0
votes
1answer
50 views

Inverse Square root of a rectangular matrix

I am trying to compute the inverse square root ($X^{-1/2}$) of a $n \times p$ matrix with $n > p$. I was wondering if we can compute it via SVD just as we do it for square diagonalizable matrices ...
1
vote
1answer
43 views

SVD : How is the normalized statistical leverage score a probability distribution?

I came across a paper related to the Singular Value Decomposition (SVD) where they calculate a normalized statistical leverage score for each column of a $m \times n$ matrix, It is defined as follows: ...
0
votes
1answer
72 views

Singular Value Decomposition of Rank 1 matrix

I am trying to understand singular value decomposition. I get the general definition and how to solve for the singular values of form the SVD of a given matrix however, I came across the following ...