Why not use the identity matrix instead of the Kronecker delta? The Kronecker delta is defined as : 
$$\delta_{mn} = \begin{cases}
1 & \text{if }m=n,\\
0 & \text{if }m\neq n.
\end{cases}$$
This is equal to the matrix $E_n$ which is a matrix with the diagonal filled with ones. 

Why not use $E_n$ instead of Kronecker delta? Does anybody see why? 

Please do tell me.
 A: The Kronecker delta is a function with domain the set of pairs (of whatever the index set is), and codomain the set $\{0,1\}$.
The $n\times n$ identity matrix, interpreted as a function, is a function with domain $n$-tuples and codomain $n$-tuples.
They are not the same. They don't have the same domain, they don't have the same codomain.
Perhaps you meant, instead:

Say that the set of indices is $\{1,2,\ldots,k\}$. Then the matrix $A$ whose $(i,j)$-entry is  $\delta_{ij}$ is exactly the $k\times k$ identity matrix. Why not use the matrix to represent the Kronecker delta function?

The answer is: for the same reason we use notation like $f(x)$ and formulas when dealing with functions, instead of using their graphs. Using the name of the function and its formula is far more elastic and useful than trying to use the "graph" all the time, which is what that matrix would correspond to. 
Added. 
It might be worth adding that I've seen some books that go the other way: they define the $n\times n$ identity matrix by saying that its $(i,j)$ entry is $\delta_{ij}$; that is, they define the identity matrix as "the graph" of the Kronecker delta function on the index set.
