Getting a feel for the transformation A on vector x which lies outside of any eigenspace In one of his videos, after 13:25 Sal starts to talk about the interpretation of the eigenvectors and how they relate to a vector $x$ being transformed by the matrix $A$.
He then goes through showing what happens when $x$ would be in one of the eigenspaces.
My questions are:


*

*what is the interpretation of the transformation being applied to a vector outside of any eigenspace, like any regular 3-dimensional $x$?

*How could I "visualize" what happens to such a vector?

*Is it related somehow to the characteristic polynomial? What is the meaning of the characteristic polynomial, beside giving us the eigenvalues at the intersections with the $x$-axis?

 A: The eigenspaces for a linear transformation $T: \mathbb{R}^n \rightarrow \mathbb{R}^n$ are the simplest building blocks for that transformation.  In fact sometimes they are too simple, in the sense that you may not be able to write $T$ as a direct sum of eigenspaces.  You can do so iff $T$ is diagonalizable.  This really is the simplest case so should be well understood first.
If your transformation is diagonalizable but not scalar -- i.e., if it is not just a single eigenspace -- then most vectors in your space will not be eigenvectors.  In general the best description I can think of for such a vector simply comes from the fact that it is a sum of eigenvectors -- indeed you can chose a basis for $\mathbb{R}^n$ of eigenvectors in this case -- so the transformation will act on it by scaling each component eigenvector by a different amount.  I would try thinking about the effect of the diagonal matrix $\left[ \begin{array}{cc} 2 & 0 \\ 0 & \frac{-1}{3} \end{array} \right]$ on a random vector $[x,y]^T$ and drawing some pictures to try to get comfortable.
When there is no basis of eigenvectors things get more complicated.  There are really two different things that could stop you from having a basis of eigenvectors.  First, over a field like $\mathbb{R}$ you simply may not have any eigenvalues at all.  The basic example of this is rotation in the plane (by any angle which is not an integer multiple of $\pi$), and this is worth thinking about.  By taking direct sums you can build linear transformations on $\mathbb{R}^{n}$ for all even $n$ without eigenvalues.  However, when $n$ is odd you must have at least one eigenvalue, because the eigenvalues are the roots of the characteristic polynomial and an odd degree polynomial over $\mathbb{R}$ must have at least one root.
However, it is possible that your characteristic polynomial factors completely into linear factors (for instance this always happens if you take $\mathbb{C}$ as your scalar field, by the Fundamental Theorem of Algebra) but that you still don't have a basis of eigenvectors.  This is about where a standard first linear algebra course tends to end, so you don't normally get a clear explanation of what is going on here but rather some rather unenlightening talk about "algebraic and geometric multiplicities".  If you go on to take a second course -- or better yet, a course in abstract algebra covering the structure theory of modules over a PID -- then you will hear the rest of the story.
It happens that I had the occasion to write a reasonably middlebrow exposition of the theory of invariant subspaces about a month ago: you can find it here.  In these notes I was influenced by Sheldon Axler's article "Down with determinants!" and the book that followed it, although in my interest in the case of a general ground field I go a bit beyond his ambitions...and at one point I explain why determinants really help a lot, as far as I can see.  But I definitely follow Axler's approach of not defining the characteristic polynomial as the determinant of $t I - T$: I agree with him that this loses all contact with geometric explanation.  In fact, the polynomial referred to in the previous sentence I call the Cayley-Hamilton Polynomial, and I give a completely different definition of the characteristic polynomial $\chi(t)$ in terms of things that are too elaborate to get into here, like primary decompositions and complete flags of invariant subspaces).  From this "geometric" definition of the characteristic polynomial it is rather straightforward to show that it is divisible by the minimal polynomial, or equivalently that $\chi(T) = 0$.  By the Cayley-Hamilton Theorem I mean the result which says that the characteristic polynomial is always equal to the Cayley-Hamilton polynomial.
If you find this exposition too brisk or abstract -- there are admittedly few examples, and if I recall I even neglected to type out some companion matrices -- then you might try Axler's book Linear Algebra Done Right instead.  He spends quite a long time telling the story, but he tells it with admirable care and insight.
A: So long as your matrix is real and dimension is 2 or 3 then some software aid is available and in 2d you can do much of it by hand. You start with a unit circle in $xy$ plane and ask what does your matrix do to each point $(x,y)$ on circle. Your circle typically maps into an ellipse. Apps such as this do the work for you.
A: I didn't have a correct definition of eigenspaces. I'm not an expert on the last part, but I'll attempt to explain anyway.


*

*If A has a limiting matrix $A^\infty$ (powers can't flip back and forth, for example), then for every (column) vector v, $A^\infty v$ is an eigenvector of A.

*So the vectors that are not in any eigenspace tend towards the eigenspaces asymptotically.

*"Meaning is not in things, but in between them." - Norman O. Brown. Matrices and polynomials both form rings. Rings are often studied geometrically by analogy with the polynomial ring, and how the 0-sets of polynomials behave best as curves, like x^2+y^2-1=0. Sending every matrix to its characteristic polynomial is a morphism between curves, called a regular map (as is sending it to that polynomial's discriminant, see this question), and the fact that it vanishes at every point of the n-by-n matrix algebra treated as the affine plane of dimension $n^2$ is proof of the Cayley-Hamilton theorem from a different perspective.
