How to find the triangle matrix of a given matrix? I read http://en.wikipedia.org/wiki/Triangular_matrix, and it says a matrix is triangularisable. But I did not find any where talking about how to do that.
E.g. how can I transform this matrix into a triangle matrix?

-2,2,3
-1,1,3
2,0,-1


 A: Usually Schur decomposition is used in numerical work for performing similarity transformations of a general matrix to a triangular matrix. The upshot is that you either have a. all the eigenvalues of your matrix are real, or b. you don't mind complex elements popping up in your triangle. (Software usually compute the "real Schur decomposition" which give a triangular matrix with bumps for matrices with complex eigenvalues.)
Alternatively, there is the Jordan decomposition, which is useful in exact arithmetic, but not too good in inexact arithmetic. Since yours is a small example, it shouldn't be too hard to get a Jordan decomposition...
A: If you find the eigenvalues $\lambda$ as the solutions to the characteristic equation
$$ 0 = \det (A - \lambda\cdot I) =  6 + 5 \lambda - 2 \lambda^2 - \lambda^3 $$
you will see that the eigenvalues of your matrix are $-3,2,-1$. Alternatively, this can be found out by running the Mathematica command
a = {{-2, 2, 3}, {-1, 1, 3}, {2, 0, -1}}
CharacteristicPolynomial[a, lambda]
d = Eigenvectors[a]
Inverse[d]

It follows that the matrix can be diagonalized - much like all matrices - so you won't need any entries above the diagonal (no nontrivial Jordan blocks will be used). It can be written as
$$ A = C D C^{-1} $$
where $D={\rm diag}(-3,2,-1)$ and the columns of $C$ are the eigenvectors corresponding to these eigenvalues. The first column is $(-1,-1,1)^T$, the second is $(3,3,2)^T$, the third is $(0,-3,2)^T$. Those can be found by solving simple sets of linear equations with the three eigenvalues found previously.
The first column of $C^{-1}$ is $(-4/5, 1/15,1/3)^T$, the second is $(2/5,2/15,-1/3)^T$, and the last column is $(3/5,1/5,0)^T$.
