How to convert Gauss-Siedel method to Successive Over Relaxation method. I am playing around with algorithms which automatically solve matrices given a matrix A and a vector b.  
I know that the Gauss-Seidel and the Successive Over Relaxation methods are similar in structure. In fact from what I understand S.O.R can be derived from Gauss-Siedel by using some number > 1 omega.  When omega = 1 in S.O.R. you would have Gauss-Siedel.  However I cannot figure out how to plug it in.
I have worked out Gauss-Seidel with the help of the wikipedia example found 
 here 
Where A and b are given, then L and U are found.  Then the inverse of L is calculated to be used to find C and T in the example.  where
$T = -D^{-1} * U$ and $C = D^{-1} * b$
Then it is simply a matter of guessing a value for $x_0$ and and multiplying the vector $x_0$ times the matrix T then adding the matrix C to the result to get $x_1$.  $x_1$ is then plugged into where $x_0$ was yielding $x_2, x_3, x_4, ...$ and so on.  When the $x_{(n-1)} = x_n$ then you are finished. (helps to see  example  I am referrencing)
basically: $x_{new} = T * x_{old} + C $
I could not however find a similar process or example (which I could comprehend at least) in regards to successive over relaxtion. Could someone please provide an example working out S.O.R. with the same matrices?  or offer how can I update this Gauss-Siedel method to be S.O.R.
 A: I would separate the matrix $A$ into $A=L+D+U$. 
Gauss-Seidel is then
$$x^{(new)}=-(D+L)^{-1}U x^{(old)}+(D+L)^{-1}b$$
To improve upon this, SOR has:
$$x^{(\omega)}=(1-\omega)x^{(old)}+\omega x^{(new)}$$
You can see that when $0<\omega<1$, the $x^{(\omega)}$ is between $x^{(old)}$ and $x^{(new)}$. When $\omega=1$, it is $x^{(new)}$. We want to overshoot a little, to make it beyond $x^{(new)}$. This is the idea of SOR.
The formula can then be derived to be:
$$D x^{(new)}=(1-\omega)Dx^{(old)}+\omega(b-Lx^{(new)}-Ux^{(old)})\\
x^{(new)}=(D+\omega L)^{-1}[(1-\omega)Dx^{(old)}-\omega U x^{(old)}]+\omega(D+\omega L)^{-1}b$$
You can either use this formula to compute the SOR result, or you can compute Gauss-Seidel result first, and use
$$x^{(\omega)}=(1-\omega)x^{(old)}+\omega x^{(new)}$$
to compute SOR, where the $x^{(new)}$ here is the result from Gauss-Seidel.
Here is also a system of equation form:
$$x_1^{(k+1)}=(1-\omega)x_1^{(k)}+\frac{\omega}{a_{11}}(b_1-a_{12}x_2^{(k)}-a_{13}x_3^{(k)}-...-a_{1n}x_n^{(k)})\\
...\\
x_i^{(k+1)}=(1-\omega)x_i^{(k)}+\frac{\omega}{a_{ii}}(b_i-a_{i1}x_1^{(k+1)}-...-a_{i,i-1}x_{i-1}^{(k+1)}-a_{i,i+1}x_{i+1}^{(k)}...-a_{in}x_n^{(k)})\\
x_n^{(k+1)}=(1-\omega)x_n^{{k}}+\frac{\omega}{a_{nn}}(b_n-a_{n1}x_1^{(k+1)}-a_{n2}x_2^{(k+1)}-...-a_{n,n-1}x_{n-1}^{(k+1)})\\$$
In that example, the Gass-Seidel would be:
$$\begin{cases}
x^{(k+1)}=\frac{1}{16}(11-3y^{(k)})\\
y^{(k+1)}=-\frac{1}{11}(13-7x^{(k+1)})\end{cases}$$
So start with $(1,1)$
$$\begin{cases}x^{(1)}=\frac{1}{16}(11-3y^{(0)})=0.5\\
y^{(1)}=-\frac{1}{11}(13-7x^{(1)})=-0.5454\end{cases}$$
$$\begin{cases}x^{(2)}=\frac{1}{16}(11-3y^{(1)})=...\\
y^{(2)}=-\frac{1}{11}(13-7x^{(2)})=...\end{cases}$$
Whereas the SOR with $\omega=1.2$ would be
$$\begin{cases}
x^{(1)}=(1-1.2)\cdot 1 +1.2\cdot 0.5=0.4\\
y^{(1)}=(1-1.2)\cdot 1+1.2\cdot (-0.5454)=-0.8545\end{cases}$$
