How to utilize the right-hand side in inverse problems Consider the inverse problem $A \, x = b$ with right-hand side $b$, using SVD:
$\qquad A = \sum s_i \, U_i \otimes V_i \ $
    — singular values $s_i, \ U_i$ and $V_i$ orthonormal bases
$\qquad x = A^+ \, b = \sum c_i \, V_i$
where $c_i \equiv filter( s_i ) \ (b \cdot U_i)
\approx {{b \cdot U_i} \over s_i}$ .
To reduce the effect of singular values that are very small or $0$,
two filter functions seem to be common:


*

*drop terms with small $s_i, \ filter( s_i ) = 0$ .
(How small is small ? see e.g.
Numerical Recipes p. 795 .)

*Tikhonov regularization
: $filter( s_i ) = { s_i \over {s_i^2 + \alpha^2} }$ .
But these don't use $b$ at all, so $x = A^+ \, b$ may be sensitive to outliers in the $b \cdot U_i$ .
Suppose we know from experience something about $b$:
for example, that it's smooth, or $\approx$ smooth + pink noise.
How can one take advantage of such information ?
Are there better filters
$\qquad filter( s_i, \, b \cdot U_i )$
that use both the $s_i$ and the right-hand side $b$ ?
Examples would be welcome.
 A: The quantity $u_i^Tb$ is a measure of the spectral content in $b$. If outliers (i.e. a "spike") are present in $b$, they "contaminate" the all coefficients, since they contain many frequencies.
If outliers are present in a single $u_i^Tb$, only one part of the total spectral content is contaminated.
Since you are proposing to use prior knowledge on $b$ being smooth, you're basically asking to use a filtered version of $b$ to remove the outliers in $b$ (not necessarily in $u_i^Tb$).
Using knowledge on $b$ being smooth, you can use a filtered version:
\begin{align}
 z &= \min_z \left\{\vert\vert{z-b}\vert\vert^2+\lambda^2\vert\vert{Dz}\vert\vert^2\right\}\\
 &=\min_z \left\{\left\lVert{\begin{pmatrix}
  I\\
  \lambda D
  \end{pmatrix} z-\begin{pmatrix}
  b\\
  0
  \end{pmatrix}}\right\rVert^2\right\}\\
 &=\min_z \left\{\left\lVert{B_\lambda z-\hat{b}}\right\rVert^2\right\}
\end{align}
Here, $D$ is for example the second derivative. The solution is given by
\begin{align}
 z = \left(B_\lambda^TB^{\,}_\lambda\right)^{-1}B_\lambda^T\hat{b}
\end{align}
You can use any filtered version for $b$ that fits your needs.
I could therefore say for a filter factor dependent on $b$:
\begin{align}
 {filter}(s_i,b) = {filter}(s_i)\frac{u_i^T\left(B_\lambda^TB^{\,}_\lambda\right)^{-1}B_\lambda^T\hat{b}}{u_i^Tb}
\end{align}
Such that
\begin{align}
c_i &=  {filter}(s_i) \frac{u_i^Tz}{u_i^Tb}u_i^Tb\\
&=  {filter}(s_i) u_i^Tz
\end{align}
Where you can see that only a filtered version of $b$ was used.
Also, filter factors may be tailored for specific applications:
"An Inverse Problem Solution Scheme for Solving the Optimization Problem of Drug-Controlled Release from Multilaminated Devices" by Zhang (https://doi.org/10.1155/2020/8380691)
