# Questions tagged [svd]

In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix, with many useful applications in signal processing and statistics.

94 questions
Filter by
Sorted by
Tagged with
4k views

### How does the SVD solve the least squares problem?

How do I prove that the least-squares solution for $$\text{minimize} \quad \|Ax-b\|_2$$ is $A^{+} b$, where $A^{+}$ is the pseudoinverse of $A$?
2k views

### What forms does the Moore-Penrose inverse take under systems with full rank, full column rank, and full row rank?

The normal form $(A'A)x = A'b$ gives a solution to the least square problem. When $A$ has full rank $x = (A'A)^{-1}A'b$ is the least square solution. How can we show that the moore-penrose solves ...
193k views

### What is the intuitive relationship between SVD and PCA?

Singular value decomposition (SVD) and principal component analysis (PCA) are two eigenvalue methods used to reduce a high-dimensional data set into fewer dimensions while retaining important ...
5k views

### Derivative (or differential) of symmetric square root of a matrix

Let A be a square, symmetric, positive-definite matrix. Let S be its symmetric square root found by a singular value decomposition. Let vech() be the half-vectorization operator. Is there a ...
914 views

8k views

### Why do positive definite symmetric matrices have the same singular values as eigenvalues?

I realize that this is because when the eigenvalues are either 0 or 1 they will have the same square root. But why does this happen?
2k views

### What is the best way to compute the pseudoinverse of a matrix?

Mathematica gives the pseudo-inverse of a matrix almost instantaneously, so I suspect it is calculating the pseudo-inverse of a matrix not by doing singular value decomposition. Since the pseudo-...
195 views

### Gradient of $A \mapsto \sigma_i (A)$

Let $A$ be an $m \times n$ matrix of rank $k \le \min(m,n)$. Then we decompose $A = USV^T$, where: $U$ is $m \times k$ is a semi-orthogonal matrix. $S$ is $k \times k$ diagonal matrix , of ...
31k views

### How can you explain the Singular Value Decomposition to Non-specialists?

I am giving a presentation in two days about a search engine I have been making the past summer, and my research involved the use of singular value decompositions, or in other words, $A=U\Sigma V^T$. ...
13k views

### Relationship between eigendecomposition and singular value decomposition

Let $A \in \mathbb{R}^{n\times n}$ be a real symmetric matrix. Please help me clear up some confusion about the relationship between the singular value decomposition of $A$ and the eigen-decomposition ...
14k views

### How is the null space related to singular value decomposition?

It is said that a matrix's null space can be derived from QR or SVD. I tried an example: $$A= \begin{bmatrix} 1&3\\ 1&2\\ 1&-1\\ 2&1\\ \end{bmatrix}$$ I'm convinced that QR (more ...
6k views

### Strang's proof of SVD and intuition behind matrices $U$ and $V$

In lecture 29 of MIT 18.06, Professor Gilbert Strang "proves" the singular value decomposition (SVD) by assuming that we can write $A = U\Sigma V^T$ and then deriving what $U$, $\Sigma$, and $V$ must ...
5k views

### Is $U=V$ in the SVD of a symmetric positive semidefinite matrix?

Consider the SVD of matrix $A$: $$A = U \Sigma V^\top$$ If $A$ is a symmetric, positive semidefinite real matrix, is there a guarantee that $U = V$? Second question (out of curiosity): what is the ...
1k views

### Continuity of an “SVD” operator

Let $A_n$ be a series of matrices, and let $A$ be another matrix. Let $S(B)$ be an SVD operator that takes a matrix and returns the left singular vectors matrix ordered by largest singular value to ...
How could one solve the following least-squares problem with Frobenius Norm and diagonal matrix constraint? $$\hat{S} := \arg \min_{S} \left\| Y - XUSV^T \right\|_{F}^{2}$$ where the $S$ is a ...