We are familiar with the theory of matrices, more specifically their eigen-theorems and associated decompositions. Indeed singular value decomposition generalizes the spectral theorem for arbitrary matrices, not just square ones. Now it only seems natural to extend this idea of 2 dimensional array of numbers to higher dimensions, i.e. tensors. But as soon as we do this, everything breaks down.
For example even the notion of rank of matrix (which we all agree to be minimum of either column rank or row rank for a matrix) seems to be conflated when it comes to tensors. The Wikipedia page seems to use degree, order and rank of a tensor synonymously (understandably due to different terminology used in different fields, but somewhat annoying nevertheless).
"The order (also degree or rank) of a tensor is the dimensionality of the array needed to represent it, or equivalently, the number of indices needed to label a component of that array."
Also for example, the very familiar concept of eigenvalues or vectors also flies out the window (though people have defined them for super-symmetric tensors). So my question is this:
What is the fundamental reason why the "nice" theorems we have in the matrix case do not extend to the case of tensors?
I can think of a couple:
- Tensors exhibit the phenomenon of rank jumping as well as field dependence; which would imply the usual rules of analysis need to be re-examined when dealing with them.
- A large class of matrices are groups, so tools from abstract algebra are available to deal with them.
- Matrices can be viewed as operators from one space to another unambiguously, whereas viewing a tensor as an operator between spaces can get confusing very quickly.
I know there are extensions to SVD for tensors, for example Tucker decomposition, HOSVD, etc; so I am not claiming it can't be done. I also understand (somewhat) that mathematicians prefer to study tensors abstractly or using differential geometry and forms. I am just curious as to why the results generalizing the concepts form the matrix case are many; what is the underlying cause for a lack of unifying framework. The above reasons seem valid roadblocks, but do they hint at something more fundamental?