The idea of eigenvectors and a transforming matrix

Question

I'm reading this tutorial on PCA: http://nyx-www.informatik.uni-bremen.de/664/1/smith_tr_02.pdf

I quote from it:

It is the nature of the transformation that the eigenvectors arise from. Imagine a transformation matrix that, when multiplied on the left, reflected vectors in the line $y=x$. Then you can see that if there were a vector that lay on the line $y=x$, it’s reflection it itself. This vector (and all multiples of it, because it wouldn’t matter how long the vector was), would be an eigenvector of that transformation matrix.

Here is the example they give:

$\left( \begin{array}{cc} 2 & 3\\ 2 & 1 \end{array}\right) \left( \begin{array}{c} 3\\ 2 \end{array}\right) = \left( \begin{array}{c} 12\\ 8 \end{array}\right) = 4 * \left( \begin{array}{c} 3\\ 2 \end{array}\right) $

I have plotted the vectors:

All I can understand is that the vector got stretched. I couldn't understand the idea of the transformation at all. Where is the transformation here?

My questions:

1- What does 'multiplied on the left' mean?

2- What does 'reflected vectors in the line $y=x$' mean?

Answer 1

Multiplied on the left simply means that you take a vector $v$ and multiply it with the matrix $A$ to get $Av$. From the left is unnecesary here, but is sometimes useful as it is also possible to mutiply a tramspose of a vector with the matrix from the right, yielding $v^TA$.

Reflecting vectors in the line $y=x$ means that you take any vector and reflect it along the line $x=y$. This creates a linear mapping of $\mathbb R^2$ into itself which can be represented by a matrix. For example, it maps $(0,1)$ into $(1,0)$, it maps $(1,2)$ into $(2,1)$ and so on. The point here is that this linear mapping has a vector, $(1,1)$ (and all its multiplies), which it maps onto itself.