# Linear independence and dependence of vectors

I am really stuck in this problem, I have only 2 days to learn matrix's base, and its generator. My problem is that I know definitions but I don't understand intuitively what they mean.

What I know: base = vectors which generate the matrix. generator = vectors which generate matrix, but there are more than enough (unlike in base)

But what is linear independence and dependence? Can you please give me trivial examples where I distinctly see the difference between them. Why dependence? What does it depend on? Why do I call (in)dependent?

Thanks in tons..

-
You use the set of generators to obtains any other vector. But sometimes you can obtain one vector from the generator set using the other vectors from the same set. this implies that the set is not composed only by linearly independent vectors. – Sigur Feb 14 '13 at 18:58
@Sigur, thanks. but what is dependence here? how can i directly see whether it is dependent or independent? – doniyor Feb 14 '13 at 19:02
Here is a good basic text: en.wikipedia.org/wiki/Linear_independence – Sigur Feb 14 '13 at 19:03
@Sigur, you helped me!. now i know what to do! – doniyor Feb 15 '13 at 4:45

Do you remember the stories of treasure maps? We were told that to get to the treasure we needed to take five steps North and three steps East. We can make this mathematical. Let $\vec{n}$ be a step North and $\vec{e}$ a vector East. Then taking five steps North and then three steps East can be written $5\vec{n}+3\vec{e}$. Going South is negative North and going West is negative East: three steps South and then one step West can be written $-3\vec{n}-\vec{e}$.

The vectors $\vec{n}$ and $\vec{e}$ span the treasure map because you can get to any point on the map by going some steps North and some steps East. (Including "negative steps".) In linear algebra speak, the vectors $\vec{n}$ and $\vec{e}$ span the treasure map. Moreover, they are linearly independent: there is no combination of North/South steps that can make up for an East/West step. You need both choices of direction to be able to reach all of the points on the map.

Let me introduce a new movement: a diagonal step $\vec{d}$. (Imagine this as a movement to the North-East direction.) Sure, $\vec{n}$, $\vec{e}$ and $\vec{d}$ still span the treasure map; $\vec{n}$ and $\vec{e}$ did that without $\vec{d}$'s help. However, these are not linearly independent. Adding $\vec{d}$ does not help me to get to points on the map that I couldn't have otherwise got to. For example, a diagonal step can be done by making a partial step to the North and a partial step to the East. Mathematically:

$$\vec{d} = \frac{1}{\sqrt{2}}\vec{n} + \frac{1}{\sqrt{2}}\vec{e} \, .$$

Generally: a set of vectors span a space if a mixture of them allows you to "get to" each point of the space. They are linearly independent if you need all of them to do that, i.e. taking one away (e.g. not being able to move North) will stop you doing this. They are linearly dependent if you have more possibilities than you need, e.g. having a diagonal step when a small North and small East step will do the job.

-
+1 great example! – Nestorghh Feb 14 '13 at 20:04
very nice example. now i got it. but i still dont know how to see this (in)dependence directly before calculating the vectors. – doniyor Feb 15 '13 at 4:35
@doniyor Being able to "see" if some vectors are linearly independent or linearly dependent is very difficult. There are some big give-aways. If you have three vectors for a two dimensional space then clearly one of them is redundant (this is the definition of dimension). If one of the vectors is a multiple of one of the others then clearly the vectors are linearly dependent (adding 2$\vec{n}$ on top of $\vec{n}$ gives nothing new). In general, with lots of vectors, it's very hard. You have to solve a set of simultaneous equations. – Fly by Night Feb 15 '13 at 17:58
@doniyor If you have, say, five vectors and each vector has, say, eight, components, then put them in to a five-by-eight matrix and find its rank. If the rank is five then they are linearly independent. I can't do this by eye. I have to do some maths. Have a good read of en.wikipedia.org/wiki/Rank_%28linear_algebra%29 – Fly by Night Feb 15 '13 at 18:05

Example of linear dependence: If $\overrightarrow{a}=\left(1,1,1\right)$, $\overrightarrow{b}=\left(2,3,4\right)$ and $\overrightarrow{c}=\left(4,5,6\right)$, then $2\overrightarrow{a}+\overrightarrow{b}=\overrightarrow{c}$. One vector can be expressed as linear combination of others, so these three vectors are linearly dependent ("value of one vector depends on values of others").

Example of linear independence: $\overrightarrow{a}=\left(1,1,2\right)$, $\overrightarrow{b}=\left(2,3,4\right)$ and $\overrightarrow{c}=\left(4,5,6\right)$ - there no vector can be expressed as linear combination of other vectors.

Also... I suppose you meant basis matrix which is formed from linearly independent vectors, using them as columns. (See more in Wiki link, given by Sigur.)

-
thanks a lot, very nice – doniyor Feb 14 '13 at 19:09

My way to think about independence is as follows:

Given the set of vectors $S = \{a,b,c\}$ Then $S$ is linearly independent when: $ma + nb + qc = 0$ if and only if the coefficients $m, n, q$ are all zero.

In other words, one of vectors $a, b$, or $c$ can't be expressed as a linear combination of the remaining vectors.

Or if we want to say this in the matrix interpretation, independent means we only have trivial solution for the coefficients.

** A simple example is the standard basis of R3: Let $S = {(1, 0, 0), (0, 1, 0), (0, 0, 1)}$

(1) Then, if I use the "number" interpretation, I should set $k1(1,0,0) + k2(0,1,0) + k3(0,0,1) = (0,0,0)$ where k1,k2,k3 are coefficients. Then by playing around with the system of equations, we see that $k1=k2=k3=0$ is the only way to make the above equation work

(2) In matrix notation, my matrix representation for the coefficients is a $3\times 3$ matrix, where the $1$'s indicate how many copies of coefficient $k1, k2$, or k3 is needed. The last column contains all 0's. This is exactly similar to what I said in (1), but it's only that we have a column vector of all zeros instead of a row vector of all zeros

$$\left[\begin{array}{ccc|c} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{array}\right]$$

Clearly by reading off the rows, we see $(0,0,0)$ is the only valid solution, and this solution is trivial.

So if you want to know where does the "independence" really happen, I guess the the word implies that the coefficients have no relation with the entries of the vectors. I don't care what numbers are in the vectors $a,b,c$. As long as I can set them into a linear combination, set it equal to the zero vector, and get zero coefficients, then even if my vectors $a,b,c$ are something crazy like (chair, cat, house), I still can exact information.

I hope this helps.

-
great, but at the end, you are calling $a,b,c$ as vectors in contrast to your 1). this confused me a bit – doniyor Feb 15 '13 at 4:30
@doniyor I change the letters to make things clear. Thanks – Cecile Feb 16 '13 at 5:32