The Convergence of Jacobi and Gauss-Seidel Iteration I have a question, is there any example that the convergence of Jacobi is faster than Gauss-Seidel ?
 A: The answer to that would according to what I know would be No. To see this, I will explain what happens in both iterative methods.
The Jacobi method tells us if there is a matrix $A=M-N,$ where in this method we denote $M=D,$ and $N=L+U,$ then the iteration matrix is :
$$B_{J}=M^{-1} N=
D^{-1}(L+U) $$ For this method to converge, the spectral radius of $B_{J}$ should be less than $1$ i.e.
$$
\rho\left(B_{J}\right)=\max _{\lambda \in \Lambda\left(B_{J}\right)}|\lambda|<1
$$
The Gauss-Siedal method tells us that if there is a matrix $A=M-N,$ where in this method we denote $M=D-L,$ and $N=U,$ then the iteration matrix $$B_{G}=
M^{-1} N=(D-L)^{-1} U $$ For the Gauss-Seidel method to converge, the spectral radius of $B_{G}$ should be less than $1,$ i.e.
$$
\rho\left(B_{G}\right)=\max _{\lambda \in \Lambda\left(B_{G}\right)}|\lambda|<1
$$
These are the necessary conditions to establish a convergence criteria. Indeed, they don't differ much except on how we choose our initial guess point and process the iteration. Now to look at what happens during the corresponding iterations of the two methods, we note that the Gauss-Siedal method is indeed faster since it operates on as much available and updated data as possible unlike the Jacobi method that relies on the updated data after the second iteration and thus takes more iterations so that the relative residual or relative error becomes less than the tolerance set. In fact the convergence rate for Gauss Siedal is twice the speed of convergence of the Jacobi method and this is true especially for strictly diagonally dominant matrices i.e.
$$|A(i, i)|>\sum_{j=1 \atop j \neq i}^{n}|A(i, j)|, \quad \forall i=1,2, \cdots, n$$
To further enhance the rate of convergence of Gauss-Siedal method, one can pick a good choice of $\omega$ in the relaxation method But the choice of the optimal $\omega$ is often difficult to make. I will not enter into further details about convergence of relaxation method but I will end my answer by attaching two interesting links on the speed of convergence of these two methods.
$(1)$ https://www.researchgate.net/publication/234783725_The_Ray_Engine/figures?lo=1
$(2)$ https://mathoverflow.net/questions/48122/the-convergence-of-jacobi-and-gauss-seidel-iteration
