Error propagation in a simple linear model (asked by a non math-major researcher). I have the following linear problem in a matrix form $Ax=Y$, where $A$ is a coefficient matrix, $x$ is a vector of unknown parameters and $Y$ is a vector of observed data. Matrix $A$ is usually over-determined and often rank deficient; it has 1000s of rows and columns. I find a solution by applying the pseudo-inverse (SVD): $x=A^{-1}Y$. I compute a conditioning number (equal about 5) to learn how the input errors propagate to the output errors. Can I determine by looking at matrix $A$ if each unknown parameter in vector $x$ is resolved well and if there is an unwanted correlation between some $x_{i}$ and $x_{j}$. I have done tests using some synthetic data and they are good, but here I would like to know if anything can be determined from the matrix $A$ alone.
 A: The covariance has the following transformation property
\begin{equation}
Cov[A\vec{x}] = A Cov[\vec{x}]A^{\intercal},
\end{equation}
where $Cov[.]$ is a covariance matrix. This property can be proved as follows
\begin{equation}
Cov[A\vec{x}] = E[(A\vec{x} - E[A\vec{x}])(A\vec{x} - E[A\vec{x}])] \\
= E[A(\vec{x} - E[\vec{x}])(\vec{x} - E[\vec{x}])A^{\intercal}] \\
= A E[(\vec{x} - E[\vec{x}])(\vec{x} - E[\vec{x}])]A^{\intercal} \\
= A Cov[\vec{x}]A^{\intercal},
\end{equation}
where $E[.]$ is the expectation.
I.e. in your case
\begin{equation}
A Cov[\vec{x}]A^{\intercal} = Cov[\vec{y}],
\end{equation}
so the covariance matrix for $\vec{x}$ has the following form
\begin{equation}
Cov[\vec{x}] = A^{+}Cov[\vec{y}](A^{\intercal})^{+},
\end{equation}
where $A^{+}$ is pseudoinverse of $A$
Sometimes even better to write the answer in terms of inverse covariance matrices:
\begin{equation}
Cov^{-1}[\vec{x}] = A^{\intercal}Cov^{-1}[\vec{y}]A.
\end{equation}
There is also well-known inequality  that links relative perturbations of the solution $\vec{x}$ and relative perturbations of the right hand side $\vec{y}$:
\begin{equation}
\frac{\|\Delta\vec{x}\|}{\|\vec{x}\|} \leq \frac{cond(A)}{1 - cond(A)\frac{\|\Delta A\|}{\|A\|}}\left(\frac{\|\Delta\vec{y}\|}{\|\vec{y}\|} + \frac{\|\Delta A\|}{\|A\|}\right),
\end{equation}
where $\Delta\vec{y}$ is the perturbation of the vector $\vec{y}$, $\Delta\vec{x}$ is the perturbation of $\vec{x}$ and $\Delta A$ is the perturbation of the matrix $A$. If the matrix perturbation is zero, the inequality has a simpler form
\begin{equation}
\frac{\|\Delta\vec{x}\|}{\|\vec{x}\|} \leq cond(A)\frac{\|\Delta\vec{y}\|}{\|\vec{y}\|}.
\end{equation}
The last two inequalities can be simply obtained using the definition of the condition number $cond(A)=\|A^{+}\|\|A\|$.
