Let's say I have an image representing a sampled function. It just so happens that I know this function can be represented as a sum of individual outer products along with some noise. So I might have an image defined as:

$I(x,y) = a_0b_0^T + a_1b_1^T + ... +\ \Omega$

Where the a and b vectors are the individual vectors forming the outer products, and $\Omega$ is a matrix of IID gaussian random variables.

If I decompose $I(x,y)$ using the SVD, I similarly get a sum of outer products:

$I(x,y) = U \Sigma V^T = s_0u_0v_0^T + s_1u_1v_1^T + ... s_Nu_Nv_N^t$

I would like to be able to extract the original image components (neglecting the noise), given these "singular images". Is there anything that I can confidently say about the relationship between my source inner products and output inner products?


The SVD is usually employed to perform denoising on an image, that is the problem that you stated: given $\tilde{I} = I + \Omega$ (where $\Omega$=noise), try to approximate $I$.

One of the possible approaches to the denoising problem is to consider a truncated SVD of the given image $\tilde{I}$, i.e.:

  1. Compute $\tilde{I} = \sum_{i=1}^n \sigma_i u_iv_i^T$
  2. Consider just the first $k$ terms $I^{(k)} = \sum_{i=1}^k \sigma_i u_iv_i^T$.

Then you have $I^{(k)} \approx I$ for appropriate choices of the truncation index $k$, that depends both on the noise magnitude $\varepsilon = \|\Omega\|$ and on the decay rate of the $\sigma_i$'s.

Informally speaking, this process works mainly because the last singular vectors are the most noisy ones. I don't know a formal explanation of this fact, and I guess that it could be truly involved.

An apparently good reference I found by googling "svd + denoising" is this one.

  • $\begingroup$ I think that's not quite what I want. I certainly want $I$ but I want it broken out like the original outer products. That's the thing that's tripping me up is if there's necessarily a relationship there. $\endgroup$ – gct Mar 8 '14 at 2:39
  • $\begingroup$ Sometimes there can be quite a lot of information in the last components of the SVD, see stats.stackexchange.com/questions/177102/… $\endgroup$ – kjetil b halvorsen Nov 3 '15 at 20:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.