Why is the Jacobi method defined the way it is? I'm just reading through some class notes, and the Jacobi method is derived (roughly) as such:
Let $A=D-L-U$, where D is the diagonal component of A, L is the negative of the lower triangular component of A, and U is the negative of the upper triangular component of A. So then,
$Ax=b \rightarrow (D-L-U)x=b \rightarrow Dx-Lx-Ux=b \rightarrow Dx=(L-U)x+b$
So the Jacobi method is defined as
Choose $x_0$ and then solve 
$Dx_{k+1}=(L+U)x_k +b$.
My question is: Why is it defined this way? Could we equally have $Ux_{k+1}=(L+D)x_k +b$, where we define A as $A=U-D-L$? I understand that it's important that $det(D) \neq 0$. Is this more likely in a diagonal matrix? Could we not alter the method depending on the matrix A we have? If for example $det(D)=0$, but we notice that $det(U) \neq 0$, could we then alter our method?
 A: It's easy to lose sight of the simple, clear intuition behind the Jacobi method when it's expressed using matrix notation.  Here's the idea. Suppose we want to solve the linear system
\begin{align}
a_{11} x_1 + \cdots + a_{1n} x_n &= b_1 \\
& \vdots \\
a_{n1} x_1 + \cdots + a_{nn} x_n &= b_n,
\end{align}
and suppose that our current best guess for the solution is
$$
x^k = \begin{bmatrix} x_1^k \\ \vdots \\ x_n^k \end{bmatrix}.
$$
A very simple idea is to solve the first equation for $x_1$, and use our most recent estimate for the other components of $x$, like this:
$$
x_1^{k+1} = -\frac{a_{12}}{a_{11}} x_2^k - \cdots - \frac{a_{1n}}{a_{11}} x_n^k.
$$
We compute $x_2^{k+1},\ldots, x_n^{k+1}$ similarly (and we can compute them all in parallel). That's all the Jacobi method is.  You can also see how a similar strategy leads to the Gauss-Seidel method.
A: The Jacobi method is part of a larger class of "matrix splitting" methods. Allow me to explain...
Splitting methods
Suppose we can write $A$ as a difference of matrices $A=M-N$ such that $M$ is nonsingular.
We refer to $M$ and $N$ as a splitting of the matrix $A$.
It follows that for any $x$ satisfying $Ax=b$,
$$
x=M^{-1}b+M^{-1}Nx.
$$
This suggests defining the so-called splitting method
$$
x_{k+1}=M^{-1}b+M^{-1}Nx_{k}.
$$
Generally, we want to choose the splitting such that


*

*$M$ is easy to invert (computationally) and

*the spectral radius of $M^{-1}N$ is strictly less than one.


The latter is a necessary and sufficient condition required to ensure that the method is convergent.
In fact, the smaller the spectral radius, the faster we expect the method to converge.
Jacobi method
One choice of splitting is the Jacobi method, in which we take $M=D$ and $N=L+U$ as suggested in your post.
This choice is motivated by the fact that when $M$ is invertible, its inverse is trivial to compute.
We have the following result for the Jacobi method:
Theorem. If $A$ is strictly diagonally dominant ($|a_{ii}|>\sum_{j\neq i}|a_{ij}|$), the Jacobi method converges.
Proof.
$$
\rho(D^{-1}\left(L+U\right)) \leq
\left\Vert D^{-1}(L+U)\right\Vert _{\infty}
=\max_{i}\frac{\sum_{_{j\neq i}}\left|a_{ij}\right|}{\left|a_{ii}\right|}
<1.
$$
Other splittings


*

*Gauss-Seidel method: $M=D-L$ and $N=U$.

*Successive over-relaxation (SOR): $M=D/\omega-L$ and $N=(1-1/\omega)D-U$ where $\omega>1$.


Remark: note that Gauss-Seidel is just a special case of SOR (take $\omega = 1$).
