Exact meaning of "Not every matrix is a tensor". I've recently begun reading about tensors and am trying to understand the second order variety in the context of euclidean $\mathbb{R}^n$ with orthonormal basis {$e_1, e_2,\ldots, e_n$}.  This seems like a simple starting point that leaves topics like covariance and contravariance for another day.  My question is w.r.t. this limited context.
Concerning the relationship between n$\times n$ matrices and second order tensors, I've read that "not every matrix is a tensor" and I'm trying to find a concise statement of which n$\times n$ matrices are second order tensors. Is it true that every n$\times n$ matrix $M$ is a tensor if and only if: 
$$M_{ij}=S_{ij}e_i\otimes e_j$$ and $S_{ij} \in \mathbb{R}$
If "yes", is it true that there aren't any other constraints on $S$ in order for $M$ to be considered a tensor?
 A: In physics, the terms "vector" and more generally "tensor", refers to an object that transforms like a vector/tensor.
First of all, a matrix is just a representation of a tensor in a certain coordinate frame, the tensor itself is an abstract notion, independent of notation and choice of coordinates. Secondly, a tensor in physics is an object, that, when expressed in another coordinate frame, takes the form
$$T'_{ij}=P_{ik}P_{jl}T_{kl}$$
if matrix $P$ takes the original frame to the primed (') frame.
In this sense general nonsquare matrices aren't tensors, nor are the matrices that represent just some sets of linear equations instead of objects that are bound to geometry of the physical space.
Tensors are very commonly symmetric (Hermitian).
Moreover, we are talking here about 2nd rank tensors. A tensor can be of any dimension (e.g. tensor of 1st rank is a vector). For instance, force is a vector, but a set of three variables is not, in the context of physics, called a vector. Even magnetic field is not strictly speaking a vector, when you go to relativistic space-time representation because it doesn't transform as one (plus, its parity is wrong under coordinate transformations that invert the space, so even classically, it's called an pseudo vector).
This is a physicists' interpretation of the question what is a tensor and what is not.
If there are any questions, I can elaborate more.
A: I think you have the relationship between matrices and tensors wrong. The tensor $e_i\otimes e_j$ corresponds directly to a matrix (rather than to an entry of a matrix), specifically the matrix $e_ie_j^T$. I am also assuming that by tensor you mean elementary tensor product, i.e. a tensor product of the form $v\otimes w$ for some $v,w\in\mathbb R^n$.
In general, $v\otimes w$ corresponds to the matrix $vw^T$. It is easy to see that this matrix has rank $1$. In fact, the converse is also true: if $M$ has rank $1$ then $M=vw^T$ for some vectors $v,w\in\mathbb R^n$. To see this, let $v$ be a $\mathrm{im}(M)$ and $w$ be a basis for $\ker(M)^\perp$. Then $vw^T=M$ up to a constant factor.
