Is it a "standard" Math/Numerical-Analysis hack to add a relatively small number e.g. 1*10E-5 to the diagonal of a squared matrix to ensure LU Decomposition (or whichever decomposition algorithm is applicable)? As opposed to "partially/totally pivoting"?

[Edit] The problem: I am computing the regressors for a statistical regression model, and the constraints are that we can no longer bring in any open source software. Hence I am drawing on my undergrad courses in Linear Algebra and Numerical Methods to implement the solution.

I am currently using Croute's LU Decomposition algorithm (http://en.wikipedia.org/wiki/Crout_matrix_decomposition) with partial pivoting on the rows.

  • $\begingroup$ I would say that one is instructive to teaching, while the other is instructive to solving. $\endgroup$ – Amzoti Mar 11 '13 at 15:08
  • $\begingroup$ I'm a programmer and only concerned with solving :) $\endgroup$ – Reuben Peter-Paul Mar 11 '13 at 15:09
  • 2
    $\begingroup$ This sounds like a really bad idea. Take the matrix $\begin{bmatrix}0&1\\1&0\end{bmatrix}$ and compare the LU decompositions you get with pivoting and with your approach without pivoting. $\endgroup$ – Rahul Mar 11 '13 at 15:19
  • $\begingroup$ I agree @RahulNarain, I also fear the negative effects that such a hack has on the model itself. $\endgroup$ – Reuben Peter-Paul Mar 11 '13 at 15:27
  • $\begingroup$ Also consider $\begin{bmatrix}-10^{-5}&1\\1&-10^{-5}\end{bmatrix}$... $\endgroup$ – Rahul Mar 11 '13 at 16:10

For the LU decomposition to happen, you need the leading principal minors of every order to be non zero.

Since the eighenvalues of the matrix $A$ are really close to the ones of the matrix $B = A+\alpha I$ for $\alpha$ close to $0$, using $B$ instead of $A$ as a first step in the decomposition, helps preventing errors at a minimum cost.


It has been my experience that solving problems using numerical methods always brings into focus the great difficulties with having general solutions that always work.

It seems that the problems always have pathological tendencies that make the solver not work in all cases.

This requires that our methods have all the tricks we can muster in the toolbox in order to handle all of these pathological cases.

Although I agree that pivoting it likely best, there may be instances where both are required in order to solve a particular problem.

This indeed is the danger with all of the math SW we see out there because one has to have some level of understanding of what can and does go wrong.

I am not sure if this answer is satisfying.


  • $\begingroup$ Thank you for your answer @amzoti. $\endgroup$ – Reuben Peter-Paul Mar 12 '13 at 14:46
  • $\begingroup$ @ReubenPeter-Paul: You are very welcome. Always remember, the best libraries have anticipated as many of the pathological things that can happen and can account for them without operator intervention! Regards $\endgroup$ – Amzoti Mar 12 '13 at 14:47
  • $\begingroup$ Great advice, Amzoti! $\endgroup$ – Namaste Apr 21 '13 at 0:28
  • $\begingroup$ @amWhy: thank you my friend! $\endgroup$ – Amzoti Apr 21 '13 at 0:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.