# What is happening in a linear algebra computation?

About a year ago I took a Linear Algebra class that was required for my degree. Unfortunately that class had an unidentified pre-requisite and started at a much higher level then I really needed. Going in I had no prior experience with linear algebra. I can definitely see how understanding linear algebra would be a very good thing to have in my field so I've been trying to piece it together ever since and have felt like I'm close but I just don't quite get it. I understand that linear algebra is a way to solve a lot of equations rapidly... In my mind this seems like that means finding values for the variables... but it didn't seem like we ever did... Instead we were doing things like multiplication of matrices and that made no sense. Or we would apply advance algorithms to get the matrix into certain forms which the reason for never made sense to me. So what does it mean to solve a system of equations? What are some real world examples that might make understanding linear algebra easier? Why are orthogonal and other types of matrices so special? Any insights, examples, suggestions are greatly appreciated!

-
As the question stands, it is way to vast to tackle in a single post. (Although I have seen heroic answers to vast questions before.) Have you already tried reading the wikipedia page on linear algebra? Also this page should deal with your question about solving equations and its relationship to linear algebra. –  Raskolnikov Mar 12 '11 at 19:08

I'm not going to try and answer all of your questions because it's really a very broad question. I will at least give you a start and then the best thing you can do is either take another course or open a book and learn some things, coming back to ask more specific questions along the way.

Surely you are familiar with equations like $5x = 6$ where we want to find all solutions $x$ where $x$ is in some field, say in this case the real numbers. In this case every nonzero element has an inverse so you can multiply by $1/5$ to get $x = 5/6$.

Although linear algebra isn't really just about solving equations, that's where it starts. It's called linear because we only want to solve equations that are linear in the unknown variable. The simplest case would be something like

$$x + y = 4, 2x - y = -1$$

Actually we can just write this in matrix form. If you remember how to multiply a matrix, then we can write this system as $Ax = b$:

$$\begin{pmatrix} 1 & 1\\ 2 & -1 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 4\\ -1 \end{pmatrix}$$

The reason why we multiply matrices is because we want to solve $Ax = b$ by multiplying by the inverse of $A$ to get $x = A^{-1}b$. Of course, not all matrices have inverses. So the set of all $n\times n$ matrices, with addition and multiplication is a ring but not a field. A ring is sort of like a field but now we remove the requirement where inverses exist for all nonzero elements. Also matrix multiplication is not commutative: $AB$ is not necessarily equal to $BA$.

In the above case $A$ does have an inverse, and you can multiply on the left by $A^{-1}$ (see if you can find it) to get the solution to this system of equations.

Thus we have found: multiplication of matrices helps us solve equations.

However, we are only beginning because finding the inverse of a matrix is tricky, so we study the different ways to represent matrices and calculate with matrices in order to more efficiently move them around. This is a bit vague but intentionally so since there is so much mathematics going on in the background which you need to learn.

Linear algebra is really about vector spaces. To appreciate the idea of a vector space you should first get some experience with abstraction by doing hundreds of problems. A vector space is just a set of elements together with addition and scalar multiplication that satisfy certain axioms. It turns out that matrices correspond to maps between vector spaces in a chosen basis of that space. This may not make too much sense to you now, but the important point is that putting matrices in different forms corresponds to changing the basis of the vector space in different ways.

The reason why we like to use vector spaces is because then we can concentrate on the algebraic properties of vector spaces without having to worry about specific numbers or equations, which then can be applied to all sorts of problems which have little do with solving equations.

The best thing you can do to understand linear algebra is to take a course/read a book and just start solving problems. It is impossible to really understand what it is about first and then practice doing it. The understanding comes with the practice.

-
Very inspiring answer! –  M Turgeon Aug 29 '12 at 20:26

Linear algebra is not necessarily about solving linear equations rapidly. Indeed, most algebraist don't care for speed except for mathematicians in numerics and computer algebra.

Linear Algebra is taught because of two dual reasons (i) it is best understood (ii) it is omnipresent within any higher mathematics.

So, constructs from linear de facto appear in any science that does anything beyond simple pen-and-paper-computation. Most prominent examples where linear algebra is used 'immediatly' compromise computer graphics and modeling biological/chemical/social systems, not to forget linear optimization or numerical linear algebra.

In further answer would (probably) end up in a huge, long list.

-