Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

A linear inverse problem is given by:

$\ \mathbf{d}=\mathbf{A}\mathbf{m}+\mathbf{e}$

where d: observed data, A: theory operator, m: unknown model and e: error.

The Least Square Error (LSE) model estimate is given by:

$\ \mathbf{\tilde{m}}=(\mathbf{A^\top A})^{-1}\mathbf{A^\top d}$

Typically the linear equation is ill-conditioned and as a consequence the model estimate tends to oscillate.

The LSE solution to the linear equation is equivalent to minimizing the following cost function:

$\ C=\|\mathbf{d-Am}\|^2$

In order to stabilize the solution we may introduce a "dampening" term:

$\ C=\|\mathbf{d-Am}\|^2 + \|\mathbf{\Gamma m}\|^2$

where $\ \Gamma$ is called the Tikhonov matrix.

The Tikhonov matrix should penalize unwanted aspect of a solution.

The stabilized LSE model estimate then becomes:

$\ \mathbf{\tilde{m}}=(\mathbf{A^\top A+\Gamma^\top \Gamma})^{-1}\mathbf{A^\top d}$

As an example problem say I have the model $\ m(x)=x^2, 0 \leq x \leq 1 $.

Instead of observing the model directly I am observing the derivative of the model + some noise (n):

$\ d(x)=\frac{dm}{dx}+e$

From measurements of d(x) I want to estimate m(x). Say I know that m(0)=0.

My linear inverse problem is now:

$\ \mathbf{d}=\mathbf{A}\mathbf{m}+\mathbf{e}, \mathbf{A=D_x}$

where $\ \mathbf{D_x}$ is the differential operator (matrix) for which I use central differences.

The LSE solution to this problem oscillates a lot: enter image description here

Therefore I want to dampen it by using an appropriate Tikhonov matrix.

Which Tikhonov matrix should I use?

share|cite|improve this question
Have you tried penalizing large variations in your solution by using a differentiation matrix as $\Gamma$? – begeistzwerst Jul 5 '12 at 9:57
Thanks begeistzwerst! Would this not "penalize" not flat estimations? I tried it and it seems to drag the solution towards a horizontal line. – Andy Jul 5 '12 at 11:00
up vote 1 down vote accepted

Try $\Gamma = \alpha \Delta$, where $\Delta$ is a discretized version of the Laplace operator (ie, a second difference operator), and $\alpha$ is a scalar regularization parameter. The action of the Laplacean is to amplify high frequency modes, so such regularization will smooth your solution.

share|cite|improve this answer
Thanks Nick! This would "penalize" deviations from a straight line? The Laplace operator: [1 -2 1] only look at the immediate neigbours though. Is there some convenient way to look at more neighbour points thereby requiring that the estimate is smooth on a larger scale? – Andy Jul 5 '12 at 10:44
I just tried the Laplace operator to a power (15) and that seemed to work quite well. The higher power of the Laplace operator the more neighbours are involved in the calculation it seems. – Andy Jul 5 '12 at 10:57
Not sure about he reasoning behind this though. Laplacian to the power of n would equal the 2*n th derivative? Since what I am looking for is smoothness on a larger scale; maybe I could use an operator like [1 0 0 0 -2 0 0 0 1] instead? – Andy Jul 5 '12 at 11:05
The laplacean's eigenfunctions are like sinusoids of higher and higher frequency, with linearly increasing eigenvalues. If you take powers of the laplacean, the eigenfunctions stay the same but the eigenvalues get raised to that power, so now the eigenvalues are increasing quadratically/cubically/whatever instead of linearly. If you want to penalize higher frequency modes more, then raising the laplacean to a power will do that. However you might also try increasing the scalar regularization parameter $\alpha$ out front instead. – Nick Alger Jul 5 '12 at 11:12

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.