How to define a linear transformation on a sub-space of a matrix. Consider we have an $n \times n$ matrix, $A$. This matrix represents a linear function from $\Bbb R^n$ to $\Bbb R^n$. Let's say we found a sub-space spanned by the vectors $u_1, u_2, \dots u_k$ where $k<n$ such that $A$ maps any vector from the span of $u_1, u_2, \dots u_k$ back to this same span. Now, we want to define a linear transformation that does exactly what $A$ does but defined from the span of $u_1, u_2, \dots u_k$ to span $u_1, u_2, \dots u_k$.
I read in a proof I'm trying to follow that the standard way to do this is to define a transformation matrix, $A'$ that is $k \times k$ and its columns are the vectors $A u_j \forall j \in 1 \dots k$, expressed in the coordinate system of $u_1, u_2 \dots u_k$. This leads to the conclusion that:
$$A' = U^T A U$$
Is this statement correct? And if so, any intuition for why its true and how do I prove it?
 A: The matrix can't have $k$ columns as multiplication with a vector in $R^n$ won't make sense. Thus, we will consider the matrix where the first $k$ columns consist of the basis of $U$.
Let $A=\begin{bmatrix} u_1 & u_2 & \dots &u_k & \dots &0\end{bmatrix}$. The columns of this matrix consist of the basis vectors of $U$. Let $v=\begin{bmatrix}x_1 \\ \vdots \\ x_n \end{bmatrix}$ be a vector in $U$.
Recall that $$\begin{bmatrix}A_{1,1} & \cdots &A_{1,n} \\ \vdots \\ A_{m,1} & \cdots & A_{m,n} \end{bmatrix}\begin{bmatrix} x_1 \\ \vdots \\ x_n\end{bmatrix}=x_1\begin{bmatrix}A_{1,1}\\ \vdots \\ A_{m,1} \end{bmatrix}+\cdots +x_n\begin{bmatrix}A_{1,n}\\ \vdots \\ A_{m,n} \end{bmatrix}$$
Now the product $$Av=x_1[u_1]+\cdots +x_k[u_k]+\cdots +x_n[0]$$
The resulting vector is written as a linear combination of the basis $u_1,\dots,u_k$ and thus $Av\in U$ and equivalently it is in span$(u_1,\dots,u_k)$.
A: Let $W\subseteq\mathbb{C}^n$ be the subspace spanned by $\{u_1,\ldots, u_k\}$, such that $Ax \in W$ for all $x\in W$. Such a subspace is called $A$-invariant.
WLOG assume that $\{u_1,\ldots,u_k\}$ forms an orthonormal basis for $W$ (otherwise apply Gram-Schmidt), and let $S = \begin{pmatrix}u_1 &\cdots & u_k\end{pmatrix}$. Since $W$ is $A$-invariant, $Au_j \in W$ is linear combination of $\{u_1,\ldots,u_k\}$ for each $j=1,\ldots,k$, so there is a $A'\in M_k$, such that $AS = SA'$. Note also that $S^*S=I_k$, because of orthonormality, where $^*$is the conjugate transpose. Thus $S^*AS=A'$.
