What do we actually rotate with rotational matrices We just learned about rotational matrices in a more theoretical context, but we only "glanced" at them.
My question: What exactly can a matrix rotate (vectors, other matrices, etc...?) and I still don't really get it how it actually rotates another object. Can somebody please give me a simple explanation (it can also be rigorous).
 A: To begin with, here is a bit of background about how you should think about linear transformations in general. Every $2 \times 2$ matrix has two pieces of information encoded in it: where the $\underline{i}$ vector lands, and where the $\underline{j}$ vector lands. Thus, in the below matrix, we know that $\underline{i}=(1,0)$ is mapped to $(a,b)$, and $\underline{j}=(0,1)$ is mapped to $(c,d)$.
\begin{bmatrix}
a & c \\
b & d \\
\end{bmatrix}
Linear transformations are essentially transformations that can be described entirely by where the $\underline{i}$ and $\underline{j}$ vectors land. If you multiply the above matrix with a $2 \times 1$ matrix, then you get
$$
\begin{bmatrix}
a & c \\
b & d \\
\end{bmatrix}
\begin{bmatrix}
x \\
y
\end{bmatrix}
=
x
\begin{bmatrix}
a \\
b
\end{bmatrix}
+
y
\begin{bmatrix}
c \\
d
\end{bmatrix}
=
\begin{bmatrix}
ax + cy \\
bx + dy
\end{bmatrix}
$$
Since the $\underline{i}$ vector is mapped to $(a,b)$, let $\underline{i_{\text{new}}}=(a,b)$. Similarly, let $\underline{j_{\text{new}}}=(c,d)$. What I'm trying to emphasise here is that when you do any linear transformation, you move the basis vectors $\underline{i}$ and $\underline{j}$  to $\underline{i_{\text{new}}}$ and $\underline{j_{\text{new}}}$. The point $(x,y)=x\underline{i}+y\underline{j}$ is mapped to $x\underline{i_{\text{new}}} + y\underline{j_{\text{new}}}$, hence why each linear transformation can be described entirely by where the $\underline{i}$ and $\underline{j}$ vectors land. Notice how $(x,y)$, which is a linear combination of the $\underline{i}$ and $\underline{j}$ vectors, is mapped to the same linear combination of $\underline{i_{\text{new}}}$ and $\underline{j_{\text{new}}}$.

Now, let's consider the rotation matrix $R_{\theta}$ that represents a rotation of $\theta$ radians anticlockwise about the origin. $\cos\theta$ and $\sin\theta$ are, by definition, the $x$- and $y$-coordinates on a circle that you land on after making an angle of $\theta$ with the positive $x$-axis, meaning that
$$
\underline{i_{\text{new}}}=(\cos\theta,\sin\theta) \, .
$$
Furthermore,
$$
\underline{j_{\text{new}}}=(\cos(\theta + \pi/2),\sin(\theta + \pi/2)) \, ,
$$
as the $\underline{j}$ vector is at right angles to the $\underline{i}$ vector. Since the derivatives of $\cos\theta$ and $\sin\theta$ are, respectively, $\cos(\theta + \pi/2)$ and $\sin(\theta + \pi/2)$, the formula for $\underline{j_{\text{new}}}$ can be simplified to
$$
\underline{j_{\text{new}}}=(-\sin\theta,\cos\theta) \, .
$$
Thus, $R_{\theta}$ is given by
$$
R_{\theta} = 
\begin{bmatrix}
\cos \theta & -\sin\theta \\
\sin \theta & \cos\theta
\end{bmatrix}
$$
Since rotation is a linear transformation, we can describe its effects by considering where the $\underline{i}$ and $\underline{j}$ vectors land. This means that we can rotate any point $(x,y)$ by $\theta$ radians about the origin by multiplying $R_{\theta}$ with the vector $x\underline{i} + y\underline{j}$:
$$
\begin{bmatrix}
\cos \theta & -\sin\theta \\
\sin \theta & \cos\theta
\end{bmatrix}
\begin{bmatrix}
x \\
y
\end{bmatrix}
=
x
\begin{bmatrix}
\cos \theta \\
\sin \theta
\end{bmatrix}
+
y
\begin{bmatrix}
-\sin\theta \\
\cos \theta
\end{bmatrix}
=
\begin{bmatrix}
x\cos\theta - y\sin \theta \\
x\sin \theta + y\cos \theta 
\end{bmatrix}
$$
For an example of when this might be useful, consider the 'negative reciprocal' rule: if a line has a gradient of $m$, then the line perpendicular to that line has a gradient of $-1/m$. This can be proven using matrices. Consider the line $l_1 $that joins the origin with the point $(x,y)$, and thus has a gradient of $y/x$. The line $l_2$, which is perpendicular to $l_1$, makes an angle of $\pi/2$ radians with $l_1$. Hence, we can map $l_1$ to $l_2$ using the rotation matrix
$$
R_{\pi/2} =
\begin{bmatrix}
0 & -1 \\
1 & 0
\end{bmatrix}
$$
Again, the entries of the above matrix can be worked out by considering where the basis vectors land. Now, if we perform a $\pi/2$ rotation on $(x,y)$, look at what happens:
$$
\begin{bmatrix}
0 & -1 \\
1 & 0
\end{bmatrix}
\begin{bmatrix}
x \\
y
\end{bmatrix}
=
x
\begin{bmatrix}
0 \\
1
\end{bmatrix}
+
y
\begin{bmatrix}
-1 \\
0
\end{bmatrix}
=
\begin{bmatrix}
- y \\
x
\end{bmatrix}
$$
The point $(x,y)$ is mapped to $(-y,x)$, which has a gradient of $-x/y$. Of course, this only proves the rule when the two lines intersect at the origin, but it's easy to visualise that wherever the two lines intersect, this negative reciporocal relationship holds.
For a more sophisticated application of rotation matrices, you can use them to prove the addition formulae for $\cos$ and $\sin$:
\begin{align}
\cos(\alpha + \beta) &= \cos\alpha\cos\beta - \sin\alpha\sin\beta \\
\sin(\alpha + \beta) &= \sin\alpha\cos\beta + \cos\alpha\sin\beta \, .
\end{align}
How? Well, for any two $2 \times 2$ matrices $A$ and $B$, their product $AB$ is the linear transformation described by applying $B$, then applying $A$. The order of operations is similar to function notation, where $f \circ g$ means 'apply $g$, then apply $f$'. Clearly then, $R_{\alpha}R_{\beta} = R_{\alpha + \beta}$, since a rotation of $\beta$ radians, followed by an alpha of $\alpha$ radians, is surely the same as a rotation by $\alpha + \beta$ radians. But, if we multiply out $R_{\alpha}$ and $R_{\beta}$ the long way, we obtain formulae for these rotations written in terms of $\cos\alpha$, $\sin\alpha$, $\cos\beta$, and $\sin\beta$; equating these two leave you with the addition formulae. I'll leave this for you to work out. Please let me know how you get on, and I'll help you with any questions you have.
A: Here' s a concrete example:
The rotation matrix
$$R_\phi := \begin{pmatrix}\cos(\phi) & -\sin(\phi) \\ \sin(\phi) & \cos(\phi) \end{pmatrix} $$
applied to the vector $(r\cos(\theta), r\sin(\theta))^T$ yields
$$\begin{pmatrix}\cos(\phi) & -\sin(\phi) \\ \sin(\phi) & \cos(\phi) \end{pmatrix} * \begin{pmatrix} r\cos(\theta) \\ r\sin(\theta) \end{pmatrix} = \begin{pmatrix} r\cos(\phi)\cos(\theta) - r\sin(\phi)\sin(\theta) \\ r\sin(\phi)\cos(\theta) + r\cos(\phi)\sin(\theta) \end{pmatrix} = \begin{pmatrix} r\cos(\phi + \theta) \\ r\sin(\phi + \theta) \end{pmatrix}.$$
That is, the original vector has been rotated counterclockwise through an angle of $\phi$.
A: Answering to " ..it can rotate also polynomial functions?"
Since $\bf R_{\varphi}$ is orthogonal , it preserves the dot product, hence the distances and the angles.
so if you apply it to the vector
$$
\left( {\matrix{ x  \cr  {f(x)}  \cr  } } \right)\quad
 or\quad \left( {\matrix{  {x(t)}  \cr {y(t)}  \cr } } \right)
$$
you get a rotated representation of the graph of $y(x)$ or of the curve $(x(t),y(t))$ which preserves the distances (lengths, derivatives,angles, integrals, ..).
