# How do the iterative methods (Jacobi and Gauss-Seidel)work?

Jacobi and Gauss-Seidel

1) How does assuming any point as an initial approximation narrow down to the correct solutions?

2) why should the coefficient matrix be diagonally dominant?

3) if I were to think of such a method, what would my path of reasoning be?

Here's a similar question, but I don't quite get the answers. I specifically don't understand what the first two answers, assume to be evident,

which "naturally" defines a fixed-point iteration

• @WlodAA to be honest, I have no idea, how to proceed numerically. I understand solving a system AX=b with the direct methods like using Cramer's rule, Gaussian eliminating etc. – Aravindh Vasu Feb 4 at 2:22
• Let me shoot in dark: is there the Banach fixed point theorem involved? Or perhaps, there is a dominating eigenvector? (The two can even cooperate together?). – Wlod AA Feb 4 at 2:26
• @WlodAA I've added an image, that's literally what we were taught. – Aravindh Vasu Feb 4 at 2:28
• Can you present the procedures in terms of matrices (so to speak globally, without even mentioning variables)? – Wlod AA Feb 4 at 2:35
• @WlodAA In wiki and in the similat question I've linked there's a procedure, splitting the coefficient matrix, but I couldn't understand it. – Aravindh Vasu Feb 4 at 2:46

You can do this in general for some system $$F(X)=0$$, $$F=(f_1,...,f_n)$$, $$x=(x_1,...,x_n)$$. The idea is to examine each equation and find its dominant variable, if there is one. If one can find a one-to-one dominance relationship of variables and equations, and if the dominance is strong enough, then you can solve each equation for its dominant variable. Assume that $$x_i$$ is the dominant variable in $$f_i$$.
Then one can loop through the equations in Jacobi-fashion, that is all-at-once in parallel (conceptually), $$x_i^{new}=\text{ Solve }( 0=f_i(x^{old}_{1..i-1},u,x^{old}_{i+1..n}) \text{ for } u )$$ or in Gauss-Seidel fashion, one-equation-at-a-time sequentially, replacing the computed variable value for the next equations $$x_i^{new}=\text{ Solve }(0=f_i(x^{new}_{1..i-1},u,x^{old}_{i+1..n}) \text{ for } u)$$
This gives a fixed-point method $$x^{new}=G(x^{old})$$ and all the theorems about them apply, esp. the Banach fixed-point theorem. Thus it is a sensible demand that $$G$$ be contracting locally around the solution (which poses a problem in practical application as you do not know the solution).