# Using inverse of matrix A as approximate inverse of matrix that is very close to A

Say we have two matrices, $A$ and $A'$ so that $A\approx A'$, and we have the inverse of $A$, $B$, where $AB=I$, and the inverse of $A'$ where $A'B'=I$. If we have some guarantee about how big any element in $A-A'$ is, is there some kind of guarantee that we can have on $B'-B$ so that we can say that $A'B\approx I$?

Update: the reason I want to do this is to see if there are any cases in linear regression where if you have some new training data, you won't have to invert the matrix again if your new data is very close to your old data.

• Take "small" diagonal matrix. – Jihad Dec 19 '14 at 11:43
• I don't understand, can you elaborate? – mlstudent Dec 19 '14 at 11:46
• Do you mean a guarantee that $A'B \approx I$? – Ben Grossmann Dec 19 '14 at 14:22
• Omnomnomnom yes. – mlstudent Dec 19 '14 at 14:38

Let the matrices satisfy $$\|A-A'\|<\epsilon,\quad AB=BA=I,\quad A'B'=B'A'=I,$$ then $$0=BA-B'A' = B(A-A')+(B-B')A',$$so $$B-B' = B(A'-A)B',$$ or $$d = \|B-B'\|\le \epsilon \|B\|\|B'\|\le \epsilon \|B\|(\|B\|+d),$$ hence $$d\le \frac{\epsilon \|B\|^2}{1-\epsilon \|B\|}.$$
The only condition is that your matrix norm satisifies $\|CD\|\le \|C\|\|D\|$. This is true for many different matrix norms, choose whichever fits your needs best.
Take matrix $$A = \begin{pmatrix} 10^{-n} &0 &0\\ 0 &10^{-n} &0\\ 0& 0 &10^{-n}\\ \end{pmatrix} ,\ B = \begin{pmatrix} 10^{n} &0 &0\\ 0 &10^{n} &0\\ 0& 0 &10^{n}\\ \end{pmatrix} , \\A' = \begin{pmatrix} 2\cdot10^{-n} &0 &0\\ 0 &2\cdot10^{-n} &0\\ 0& 0 &2\cdot10^{-n}\\ \end{pmatrix}$$ Then $A - A' = A$ which is very small matrix, but $A'-B\approx 10^n\cdot I$.
• That is why sometimes is more practical to estimate $\|A-A'\|/\|A\|$...=) – TZakrevskiy Dec 19 '14 at 15:25