# How to utilize the right-hand side in inverse problems

Consider the inverse problem $A \, x = b$ with right-hand side $b$, using SVD:
$\qquad A = \sum s_i \, U_i \otimes V_i \$ — singular values $s_i, \ U_i$ and $V_i$ orthonormal bases
$\qquad x = A^+ \, b = \sum c_i \, V_i$
where $c_i \equiv filter( s_i ) \ (b \cdot U_i) \approx {{b \cdot U_i} \over s_i}$ .

To reduce the effect of singular values that are very small or $0$, two filter functions seem to be common:

But these don't use $b$ at all, so $x = A^+ \, b$ may be sensitive to outliers in the $b \cdot U_i$ .

Suppose we know from experience something about $b$: for example, that it's smooth, or $\approx$ smooth + pink noise. How can one take advantage of such information ? Are there better filters
$\qquad filter( s_i, \, b \cdot U_i )$
that use both the $s_i$ and the right-hand side $b$ ?

Examples would be welcome.

The quantity $$u_i^Tb$$ is a measure of the spectral content in $$b$$. If outliers (i.e. a "spike") are present in $$b$$, they "contaminate" the all coefficients, since they contain many frequencies. If outliers are present in a single $$u_i^Tb$$, only one part of the total spectral content is contaminated. Since you are proposing to use prior knowledge on $$b$$ being smooth, you're basically asking to use a filtered version of $$b$$ to remove the outliers in $$b$$ (not necessarily in $$u_i^Tb$$). Using knowledge on $$b$$ being smooth, you can use a filtered version: \begin{align} z &= \min_z \left\{\vert\vert{z-b}\vert\vert^2+\lambda^2\vert\vert{Dz}\vert\vert^2\right\}\\ &=\min_z \left\{\left\lVert{\begin{pmatrix} I\\ \lambda D \end{pmatrix} z-\begin{pmatrix} b\\ 0 \end{pmatrix}}\right\rVert^2\right\}\\ &=\min_z \left\{\left\lVert{B_\lambda z-\hat{b}}\right\rVert^2\right\} \end{align} Here, $$D$$ is for example the second derivative. The solution is given by \begin{align} z = \left(B_\lambda^TB^{\,}_\lambda\right)^{-1}B_\lambda^T\hat{b} \end{align} You can use any filtered version for $$b$$ that fits your needs.
I could therefore say for a filter factor dependent on $$b$$: \begin{align} {filter}(s_i,b) = {filter}(s_i)\frac{u_i^T\left(B_\lambda^TB^{\,}_\lambda\right)^{-1}B_\lambda^T\hat{b}}{u_i^Tb} \end{align} Such that \begin{align} c_i &= {filter}(s_i) \frac{u_i^Tz}{u_i^Tb}u_i^Tb\\ &= {filter}(s_i) u_i^Tz \end{align} Where you can see that only a filtered version of $$b$$ was used.