Understanding matrices as linear transformations & Relationship with Gaussian Elimination and Bézout's Identity

Question

I am currently taking a intro course to abstract algebra and am revisiting ideas from linear algebra so that I can better understand examples.

When i was in undergraduate learning L.A., I thought of matrix manipulations as ways of solving $n \times n$ systems of equations. Recently i was exposed to the idea of a matrix being a linear transformation, and matrix multiplication being composition of linear transformations. Im trying to understand this in a more intuitive way and was hoping for some insight...

I was thinking of a basic $2\times2$ example and how it affects a point $(x,y)$. We could have a matrix :

\begin{bmatrix} a & b \\ c & d \end{bmatrix} When we 'apply' or multiply this to a point $(x,y)$ using matrix multiplication we get new $x' = ax + by$ and $y' = cx + dy$.

So if $b,c = 0$, then I can see that what we are doing is 'scaling' both $x \;\& \;y$. I'm guessing that if $b,c \neq 0$, then this becomes some sort of rotation or reflection, but how do you understand this on a fundamental level?

How do these operations relate to Gaussian elimination when we are trying to solve systems of equations? Or are are these two seperate applications of matrices?

Another observation is that when multiplying a matrix such as this one with a point, we get two equations which remind me of Bézout's identity. Am I overanalyzing this or can I draw connections between these two concepts?

Thanks for any input!

Answer 1

Personally, I don't feel that Gaussian elimination is the best ways to view linear transformations. It is a standard theorem that every linear mapping can be represented as a matrix under a certain basis. For simplicity, I will stick with the standard basis.

Suppose we have a linear mapping $T:\ \mathbb{F}^n \rightarrow \mathbb{F}^n$. The action of the linear mapping is entirely determined by it's action on the basis vectors. If we let $$\mathbf{v} = c_1\mathbf{e_1} + \cdots + c_n\mathbf{e_n}$$ then correspondingly $$T(\mathbf{v}) = c_1T(\mathbf{e_1}) + \cdots + c_nT(\mathbf{e_n})$$ so that knowing the set $\left\{T(\mathbf{e_1}),\ \cdots,\ T(\mathbf{e_n})\right\}$ is enough to determine the nature of the mapping.

Conversely, this property allows show to determine how a matrix acts as a mapping. If we have a matrix $$A = \begin{pmatrix}\mathbf{a_1} & \cdots & \mathbf{a_n}\end{pmatrix}$$ where $\mathbf{a_i}$ are the column vectors of $A$, then the action of the mapping induced by multiplication by $A$ will be $$A\mathbf{e_i} = \mathbf{a_i}$$ Multiplying $A$ by the $i$th standard basis vector has the effect of selecting the $i$th column of $A$. You can clearly see the action of the mapping in this way; the matrix maps the $i$th standard basis vector to it's $i$th column.

These methods give a geometric interpretation to the matrices, but it is not necessarily the most natural one. Just because we know the action of the mapping doesn't mean that we necessarily understand it. This is why the concept of diagonalization (or more generally Jordanization as tomasz mentions) is introduced. Diagonalization is the process of selecting the most geometrically natural basis for the mapping.

If the basis $\left\{\mathbf{v_i}, \cdots,\ \mathbf{v_n}\right\}$ diagonalizes $A$ to $$D = \mathrm{diag}(\lambda_1,\ \cdots,\ \lambda_n)$$ then the action of the mapping is a dilation of factor $\lambda_i$ in the direction of the $i$th basis vector, $\mathbf{v_i}$.

Answer 2

Matrices are indeed linear transformations, in the sense that they pick up a vector and then apply a rotation plus an expansion to it (in a vector space no more can be asked for, as all vectors must raise from the origin [this, viewed as geometric objects]). This idea gives birth to the concept of invariant directions, which are the ones whose vectors, once a matrix is applied to them, get expanded but not rotated. The vectors generating such subspaces are the eigenvectors associated to that matrix (and thus, to the linear map it defines), and the expansion coefficients attached to them are the eigenvalues.

Answer 3

Looking at a matrix in the standard basis, you're unlikely to have any more insight than what you have already written: it changes each of $e_1,e_2$ into a linear combination of them as dictated by the matrix. However, Jordan's theorem says that for an $n\times n$ matrix $A$ over an algebraically closed field $K$ (such as the field of complex numbers), there is a set of linearly independent vectors $v_1,\ldots, v_n$ such that for each $v_j$ it is either the case that $Av_j=\lambda_j\cdot v_j$ or $Av_j=\lambda_j\cdot v_j+v_{j-1}$, for some $\lambda_j\in K$.

The field of real numbers is not algebraically closed, but a form of Jordan's theorem still holds that says that in addition to $v_j$ as above, you may have some pairs $w_j,w_j'$ of vectors such that $A$ acts on $w_j$ by rotating and scaling it in the plane of $w_j,w_j'$, and possibly sliding it along $w_{j-1}$, and similarly with $w_j'$. In case of $2\times 2$ matrices, it just says that any such real matrix is, up to scaling, a matrix of rotation in some plane, or a matrix that scales some two vectors (not necessarily both with the same ratio), or a matrix similar to $\begin{pmatrix} \lambda & 1 \\ 0 & \lambda\end{pmatrix}$ for some real $\lambda$.

Multiplying matrices corresponds exactly to composing respective linear transformations; in general, I doubt it has to do with euclidean algorithm or Bézout's identity.

Answer 4

First, matrices can represent linear transformations as in $ x \mapsto Ax$, but they can also represent bilinear forms: $<x,y> = x^T A y$. But here we will stick to the linear transformation case. What you should do is search for some insight by looking fos some canonical examples.

Rotation matrices: these are implementing rotations. For the simplest case, the plane, we have $R=\left(\begin{smallmatrix} \cos(\theta) & -\sin(\theta) \\ \sin(\theta) & \cos(\theta) \end{smallmatrix}\right)$. You should try some examples and plot them! Then we have reflection matrices: You can find a derivation of a 2D reflection matrix here: http://www.scibuff.com/2009/06/22/reflection-matrix/ Together, rotation and reflection matrices are called orthogonal matrices. Rotations have determinant $1$, and reflections have determinant $-1$. They satisfy $Q^T Q= Q Q^T = I$ where $I$ is the identity matrix. To see they have determinant $\pm 1$, calculate $1=\det(I)=\det(Q^T Q) =\det(Q)^2$ and solve!

A very simple example is in $\mathbb R^3$, a rotation with rotation axes equal to the vertical ($z$) axes, and rotating an angle $\theta$ in the $xy$-plane: $$ \begin{pmatrix} \cos(\theta) & -\sin(\theta) & 0 \\ \sin(\theta) & \cos(\theta) & 0 \\ 0 & 0 & 1 \end{pmatrix} $$ Again, you should calculate an draw some examples!

In mechanics and engineering diciplines the shear matrices are important: Let $$ S = \begin{pmatrix} 1 & 0.5 \\ 0 & 1 \end{pmatrix} $$ This effects a shear: You should compute some examples, and you will see what it means. More examples to come ...