I’m a high school student studying Linear Algebra for Neural Networks and am on Linear Transformations and Basis Change.

When using Khan Academy, it said in regards from changing basis and linear transformations within the same dimension (Rn->Rn) that to obtain a matrix D that transforms some vector x we use this ‘formula’

$$ D = C^{-1}AC $$ Where C is the change of basis transformation matrix, and A is the original transformation matrix.

But, in the book ‘Mathematics for Machine Learning’, it speaks about transforming between dimensions $\mathbb{R^n}$ to some $\mathbb{R^m}$ and using 4 bases ”B, B’, C & C’”. Then it states to find the new transformation matrix A’ which I’m guessing is similar to what Khan Academy references as D, we use: $$ A’ = T^{-1}AS $$ Where, quoted from the book, ‘Here, S of $\Bbb{R^{nxn}}$ is the trans. Matrix of IDv that maps coordinates with respect to B’ onto coordinates with respect to B, and T of $\Bbb{R^{mxm}}$ is the trans. Matrix of IDw that maps coordinates with respect to C’ onto coordinates with respect to C”. Why are we dealing with 4 bases now?

Apologies if I’m explaining this poorly. I’d appreciate if anyone could help me on this. If you need an image of what I’m talking about in the book I can, I just didn’t want to clutter the original post too much.

Thanks, Justin

  • 1
    $\begingroup$ When we're talking about a linear transformation from $\mathbb{R}^n$ to $\mathbb{R}^n$ we typically use the same basis in both the range and domain, so there are only two bases involved. But of course, with a transformation from $\mathbb{R}^n$ to $\mathbb{R}^m$ the basis for the range is necessarily different from that of the range. That gives us two old bases and two new ones. $\endgroup$ – saulspatz Apr 30 at 21:34
  • $\begingroup$ Thanks! Sometimes that's the problem with textbooks and online resources, they assume you'll just pick up on everything they glide over (it's still a really good book). So, do the normal Basis changing and linear mapping rules apply just with a bit more complexity? Just for intuition. $\endgroup$ – Justin Cunningham Apr 30 at 21:59
  • $\begingroup$ Do you know that if $v$ is a vector in $\mathbb R^n$ described in a basis $e_i$ then $v=v_11e_1+...+v_ne_n$? but if you have a basis change, say $b_j=B_{1j}e_1+...+B_{nj}e_n$ for a new basis, then coefficients are arranged $B=[B_{ij}]$ giving an invertible matrix, and with this, do you know that the new components for $v$ can be calculated with $B^{-1}v$? With that the formulas in your questions can be proved easly. $\endgroup$ – janmarqz Apr 30 at 23:11

If you have two basis changes, say $S:\mathbb R^n\dashrightarrow\mathbb R^n$ and $T:\mathbb R^m\dashrightarrow\mathbb R^m$, where we use $\dashrightarrow$ to indicate basis change to get no confused with a linear transformation arrow, and you have a linear transformation $A:\mathbb R^n\to\mathbb R^m$, then the assignment is given by symbols $v\mapsto Av$. So, if we involve the basis changes then we have $$Av=(AS)S^{-1}v$$ which explain how the old components of $Av$ are related (via the $AS$ matrix) with the new components of $v$. But $Av=TT^{-1}Av=ASS^{-1}v$ brings the basis change $T$ into the play, then $$T^{-1}Av=(T^{-1}AS)S^{-1}v.$$ Here we can see how the new components of $Av$ under the basis change $T$ are fasten with the new components of $v$ under the change $S$ through the matrix $T^{-1}AS$.

| cite | improve this answer | |
  • $\begingroup$ Thanks for replying! Linear algebra is definitely a new way of thinking in maths compared to traditional school maths. So if I got this right, the first equation shows transforming v by matrix A and also the basis change S. Then the next equation shows transforming v by matrix A and then changing to Basis S then basis T? $\endgroup$ – Justin Cunningham May 1 at 0:23
  • $\begingroup$ One introduce $SS^{-1}$ and $TT^{-1}$ because both are the identity and they allow the algebraic trick to not alter the objects. Remember that $S^{-1}v$ gives you the description of $v$ in the new basis, and similar $T^{-1}(Av)$ gives you the description of $Av$ in the corresponding new basis $\endgroup$ – janmarqz May 1 at 0:50
  • $\begingroup$ I appreciate this. Made it so much clearer. I understand why textbooks describe things using summation notation and other abstract notation but sometimes simple explanations like these are needed instead! $\endgroup$ – Justin Cunningham May 1 at 1:40
  • $\begingroup$ the next level of simplicity would be an explicit example $\endgroup$ – janmarqz May 1 at 2:32
  • $\begingroup$ check this answer that exemplificate how the change of basis works: math.stackexchange.com/questions/1736727/vector-as-a-tensor/… $\endgroup$ – janmarqz May 1 at 2:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.