Singular Value Decomposition-noisy data I have a system of the form 
$$Ay=f,$$
where $A$ is a $N\times4$ matrix, $y$ a 4-element array of unknows and $f$ an $N$-element array. 
I add Gaussian noise in my data. I tested the following cases:


*

*$A$ noise-free and $f$ noisy

*$A$ noisy and $f$ noise-free

*$A$ and $f$ both noisy.


For the first case where there is noise only in matrix $f$ everything looks fine. For the other two cases where there is a noise in matrix $A$ I have a systematic error in my results and I can't understand why this happened. I was wondering if anyone have any idea about what is going wrong and give me a hint.
 A: Suppose you have noise in your matrix $A$. We may represent this by $A = A'+\Omega$, where $\Omega$ represents the Gaussian noise, and $A'$ represents the noise-free matrix.
Compute $A^TA$:
$$A^TA = (A'+\Omega)^T(A'+\Omega) = A'^TA'+\color{red}{\Omega^TA'}+\color{red}{A'^T\Omega} + \color{red}{\Omega^T\Omega}.$$
It should be clear that the terms in red are nonzero, and as a result the solution to the linear system will be systematically perturbed -- in essence, you're solving for a different linear system.
Contrast this to the case where $A$ is well-formed and $f$ is noisy (with noise vector $w$):
$$Ay = f'+w \\
y = (A^TA)^{-1}A^T\left(f'+w\right) = y' + \color{blue}{\left(A^TA\right)^{-1}A^Tw}.$$
In this case, the solution is simply the solution to the nominal, noiseless system $y'$ with a noise component (represented in blue) added on. Presumably, your Gaussian noise is zero mean -- this would help explain why you see no such systematic error.
A: I share the same interest in perturbation of such linear systems, a couple of years ago I stumble across the work of Per Christian Hansen in the book Discrete Inverse Problems (chapter 4). 
There, he goes through regularization:
f noisy, and perturbation:
A noisy and f noisy.
I hope that this answer helps.
