Question about iteration method (Gauss- Seidel) We want to solve the linear system $Ax=b$, where $A$ is SPD. We use a method similar to steepest decent method. 
For the first searching direction $d^1, d^2, \cdots d^n$ are chosen to be the standard unit vector $e_1, e_2,\cdots, e_n$. Then the next $n$ search directions$ d^{n+1}, d^{n+2}\cdots, d^{2n}$ are again $e_1, e_2,\cdots, e_n$. We have $x^{(k+1)}=x^{(k)}+\alpha_k d^k$, where $\alpha_k = \frac{d^k\cdot (b-Ax^{(k)})}{Ad^k\cdot d^k}$. The question is 
prove that each group of $n$ step is one iteration of Gauss Seidel method. 
I tried to express $x^{(k+1)}$ and $x^{(k)}$ as vector, but it differs to much to the formula of Gauss- Seidel method. 
 A: Let $A=(a_{ij})=L_*+U$, where $L_*$ is the lower triangular part of $A$. Let $x$ and $x^+$ be the initial and final approximation of the Gauss-Seidel method so that
$$
x^+=x+L_*^{-1}r, \quad r := b-Ax.
$$
Now consider solving $L_*c=r$, $c:=[\gamma_1,\ldots,\gamma_n]^T$. Since $L_*$ is lower triangular, we can successively solve for $\gamma_1,\ldots,\gamma_n$ and write
$$
x^+=x+\gamma_1e_1+\ldots+\gamma_ne_n.
$$
Let $x_0:=x$. We can define a recursion
$$
x_i=x_{i-1}+\gamma_ie_i, \quad i=1,\ldots,n,
$$
in which we successively update the components of the solution vector to obtain finally $x^+=x_n$. We just need to show now that $\gamma_i=e_i^T(b-Ax_{i-1})/a_{ii}$.
The coefficient $\gamma_i$ is the $i$th component of the solution of $L_*c=r$. By considering a partitioning of the leading $i\times i$ part of this system, we have
$$
\begin{bmatrix}\tilde{L}_* & 0 \\ a_i^T & a_{ii}\end{bmatrix}
\begin{bmatrix}\tilde{c}_{i-1}\\\gamma_i\end{bmatrix}=\begin{bmatrix}\ast \\ e_i^Tr\end{bmatrix},
$$
where $a_i:=[a_{i1},\ldots,a_{i,i-1}]^T$ and $\tilde{c}_{i-1}:=[\gamma_1,\ldots,\gamma_{i-1}]^T$,
so
$$
\gamma_i=\frac{1}{a_{ii}}\left(e_i^Tr-a_i^T\tilde{c}_{i-1}\right).
$$
Note that the components of $\tilde{c}_{i-1}$ are the first $i-1$ components of $x_{i-1}-x$ and the remaining components of the latter are zero. Hence $a_i^T\tilde{c}_{i-1}=e_i^TA(x_{i-1}-x)$ and 
we have
$$
\gamma_i 
= 
\frac{1}{a_{ii}}\left[
e_i^Tb-e_i^TAx-e_i^TA(x_{i-1}-x)
\right]=\frac{1}{a_{ii}}e_i^T(b-Ax_{i-1}).
$$
