Please can anybody help me with explaining, in detail, why and how the author solves the system of differential equations by such a step (see below)? How the equation 6.3 is formed step by step? Why such a column basis is valid? It would be extremely helpful if someone knows the name of the method or could give an exemplification.
The terms multiplied by $\mu$ & $\nu$ are modeling diffusion (this is a standard center difference scheme). What we know about the continuous diffusion operator (with appropriate boundary conditions) is that its eigenvectors are sines &/or cosines. If you prefer, they're exponentials with imaginary powers (the two are equivalent by Euler's formula). This is just the discrete analog: note that the vectors employed for the change of coordinates are exactly such complex exponentials evaluated at the grid points. You can check yourself that they are eigenvectors of the discretized diffusion matrix.