Relation between Jacobi and Gauss-Seidel Methods? I want to check the convergence of Jacobi (from now on J) and of Gauss-Seidel (from now on GS) methods applied to a linear system $Ax=b$, where $A$ is square and non-singular.

I was wondering if the convergence of one of the two methods shed
  light on the convergence of the other. I mean:
When can I say that if J converges then GS converges? When can I
  say that if GS converges then J converges?

I know that if $A$ is (for example, but this is true with other very similar hypothesis) strictly diagonally dominant then the spectral radius of J and GS matrices is $<1$ and so both J and GS converge. Also I know that for example if $A$ is Hermitian with real positive diagonal elements then $A$ is positive defined iff GS converges.
 A: *

*$$  \begin{pmatrix}
    1 & 0 & 1 \\
    -1 & 1 & 0 \\
    1 & 2 & -3 \\
    \end{pmatrix}
$$


is  convergent  by  J  but divergent by  GS.
and
$$
    \begin{pmatrix}
    1 & 0.5 & 0.5 \\
    0.5 & 1 & 0.5 \\
    0.5 & 0.5 & 1\\
    \end{pmatrix}
$$
is convergent by  GS  but divergent by  J.


*In general,the convergent speed by GS is faster than J if them are also convergent.

A: There is an article that addresses exactly this question. The article is found
here
under the title: 
The Convergence of Jacobi and Gauss-Seidel Iteration.
Note that Theorem 2  says that for $n=2$ Gauss-Seidel converges if and only if Jacobi converges. For $n>2$  there is no general relation. As shown bt @jean above there are examples where one converges and the other does not, and vice-versa.
If the matrix is diagonal dominant then both converge. I can point to the following important texts that address the convergence of these two methods and of iterative methods for linear systems in general. 
R. Varga. Matrix Iterative Analysis. Springer-Verlag Berlin, 2009. 
S. Venit. The convergence of Jacobi and Gauss-seidel iteration. Mathematics Magazine, 3:163–167, 1975.  (the link above)
D. M. Young. Iterative solution of large linear systems. Academic Press New York, 1971.
347
