Qualitatively, what is the difference between a matrix and a tensor? Qualitatively (or mathematically "light"), could someone describe the difference between a matrix and a tensor? I have only seen them used in the context of an undergraduate, upper level classical mechanics course, and within that context, I never understood the need to distinguish between matrices and tensors. They seemed like identical mathematical entities to me.
Just as an aside, my math background is roughly the one of a typical undergraduate physics major (minus the linear algebra).
 A: Coordinate-wise, one could say that a matrix is a "square" of numbers, while a tensor is a $n$-block of numbers. But this is horrible, not insightful and even a bit wrong, since those coordinates must "change in appropriate ways" (this is part of why this is horrible).
It may be best to think as  follows: given a vector space $V$,  a matrix can be seen in an adequate way as a bilinear map $V^* \times V \rightarrow \mathbb{R}$ (since you asked for it, I'll not enter in details. Here, $V^*$ is the dual of $V$). A tensor can be interpreted as a multilinear map $V^* \times... \times V^* \times V \times ... \times V \rightarrow \mathbb{R}$ (not necessarily the same quantity of $V^*$'s and $V$'s). 
Hence, a matrix is a kind of tensor. But tensors are more general.
A: A rank 0 tensor is a scalar.
A rank 1 tensor is a row or column vector.
A rank 2 tensor is a matrix, often square.
A rank 3 tensor? Think 3D matrix. Instead of a rectangle with data entries for each column and row, think of a cube.
Rank 4... go 4D!
A: Matrices are a special type of tensor, rank 2. Scalars, vectors, matrices, are all tensors. 
Honestly tensors are so general the vast majority of things you deal with in your class are tensors. 
