Why do we need a Jordan normal form? My professor said that the main idea of finding a Jordan normal form is to find the closest 'diagonal' matrix that is similar to a given matrix that does not have  a similar matrix that is diagonal.
I know that using a diagonal matrix is good for computations and simplifying powers of matrices. 
But what is the potential of finding a matrix with Jordan form? what is this 'almost diagonal' matrix gives me?
We learnt how to find it, without knowing what is the main idea behind it and what are the applications used with it, so I can't really understand it.
 A: Generally, if we can find a canonical form for some object, it simplifies reasoning about the object. If we want to prove some fact, we need only deal with the canonical form rather than all possibilities.
The Jordan normal form is not quite canonical (the blocks can be permuted in general) but serves the same purpose. It shows how repeated eigenvalues can affect the eigenvalue structure.
Omnomnomnom's answer shows how analytic functions can be defined,
other answers show how $\det$ and $\operatorname{tr}$ are related
to the eigenvalues.
A word of warning, in absence of special structure (or exact arithmetic), it is impossible to numerically determine the Jordan form of a matrix if there are repeated or nearby eigenvalues.
Other forms are better for numerical work, for example, the Schur decomposition allows us to write a matrix in the form $QTQ^*$, where 
$Q$ is unitary and $T$ upper triangular. This form does not reveal
the eigenspace structure as clearly as the Jordan normal form, but
can be computed in a numerically stable fashion.
A: Essentially the same that the diagonal form. It simplifies some computations, like powers,etc. If the matrix you are working with is not similar to a diagonal matrix, you settle for next best: the Jordan canonical form
A: It is demonstrated in these notes, page 29-30, that exponential of a square matrix $M$ can be explicitly computed if either $M$ is diagonalisable or nilpotent, or if you know how to decompose $M=S+N$ for a diagonal matrix $S$ and a nilpotent matrix $N$ with $SN=NS$. Such a decomposition is justified by Jordan normal form.
A: First of all, the Jordan form is also "good for computations and simplifying powers of matrices".  In particular, if
$$
J = \pmatrix{\lambda&1\\&\lambda&1\\&&\ddots & \ddots \\&&&&1\\&&&&\lambda}
$$
Then we compute (when $k>n$)
$$
J^k = \pmatrix{\lambda^k & k \lambda^{k-1} & \frac 1{2!} k(k-1)\lambda^{k-2} & \cdots & \frac 1{(n-1)!} k(k-1) \cdots(k-n + 1)\lambda^{k-n + 1}\\
0 & \ddots & \ddots & \cdots\\
\vdots & \ddots}
$$
More generally, if $f(x)$ is an analytic function (such as $f(x) = e^x$), we have
$$
f(J) = \pmatrix{f(\lambda) & f'(\lambda) & \frac 1{2!} f''(\lambda) & \cdots & \frac 1{(n-1)!} f^{(n-1)}(\lambda)\\
0 & \ddots & \ddots & \cdots\\
\vdots & \ddots}
$$
Jordan form is also important for determining whether two matrices are similar.  In particular, we can say that two matrices will be similar if they "have the same Jordan form".
A: Exercise: using Jordan canonical form, prove that a matrix is diagonalizable (over C) iff the minimal polynomial has no repeated roots. Prove the Cayley Hamilton theorem (assuming Jordan form). (The idea in all cases is to understand these theorems for a Jordan block.)  Prove that the determinant is the product of the eigenvalues, and that the trace is the sum. 
So, you will see that the Jordan canonical form summarizes a lot of theorems as trivial corollaries. The ones I listed are just what came to mind, I'm sure there are others.
By the way - you will also notice that some of these theorems are trivial for diagonal matrices. There is a technique for proving theorems for diagonal matrices, and extending them to all matrices, since diagonalizable matrices are dense in the space of matrices, and so equalities of continuous functions will carry over to the whole space (you can even prove theorems in abstract linear algebra over any ring statement this way). All matrices have a JCF, so I guess the situations when you can prove stuff by reduction to JCF are strictly larger than when you can prove by reduction to diagonalizable matrix. (For example, the first exercise I listed.)
