Intuition for Formal Definition of Linear Independence I learned about this a long time ago but it never really clicked, which led me to these questions:


*

*How the formal definition (at the bottom) works. I have a rough intuition: linear independence is where the variables are independent and don't affect each other. But I don't follow the formal definition. I would like to have a deep understanding of the formal definition based on these linear combination equations. I'm not sure how a linear combination constructed in a certain way can tell you the variables are independent or not.

*Why set the linear combination equations to $\vec{0}$. I don't see how setting to zero helps determine independence or not.

*Why choose $a_i$ to be non-zero in one case. It seems arbitrary.



From Wikipedia:

A subset $S=\{{\vec {v}}_{1},{\vec {v}}_{2},\dots ,{\vec {v}}_{n}\}$ of a vector space $V$ is linearly dependent if there exist a finite number of distinct vectors ${\vec {v}}_{1},{\vec {v}}_{2},\dots ,{\vec {v}}_{k}$ in $S$ and scalars $a_{1},a_{2},\dots ,a_{k}$, not all zero, such that
$$a_{1}{\vec {v}}_{1}+a_{2}{\vec {v}}_{2}+\cdots +a_{k}{\vec {v}}_{k}={\vec {0}}$$
where ${\vec {0}}$ denotes the zero vector.
The vectors in a set $T=\{{\vec {v}}_{1},{\vec {v}}_{2},\dots ,{\vec {v}}_{n}\}$ are linearly independent if the equation
$$a_{1}{\vec {v}}_{1}+a_{2}{\vec {v}}_{2}+\cdots +a_{n}{\vec {v}}_{n}={\vec {0}}$$
can only be satisfied by $a_{i}=0$ for $i=1,\dots ,n$.

So my understanding is, there are two subsets $S$ and $T$ of $V$. In one of them, the coefficients are not all zero, in the other they are all zero. In one case they are linearly dependent, in the other not. I don't understand why though; that's as much as I understand. Not sure why the equations were constructed like this in the first place.
 A: Here's my intuition: Think of vectors as the axes that we use to define a two-dimensional, three-dimensional, or $n$-dimensional space. A set of vectors is linearly dependent when one of the vectors isn't necessary — it doesn't add anything useful to our coordinate system.  This happens when one vector is just a linear combination of other vectors in the set.
As an example, consider these three vectors in two-dimensional space:
$\vec w = \begin{bmatrix}2 \\3 \end{bmatrix}$,
$\vec x = \begin{bmatrix}1 \\0 \end{bmatrix}$,
$\vec y = \begin{bmatrix}0 \\1 \end{bmatrix}$
In this case, we can express $\vec w$ as a linear combination of the other two vectors, $\vec w = 2 \vec x + 3 \vec y$:
$\begin{bmatrix}2 \\3 \end{bmatrix} = 2\cdot \begin{bmatrix}1 \\0 \end{bmatrix} + 3\cdot \begin{bmatrix}0 \\1 \end{bmatrix}$
Equivalently, we can rewrite this as $2\vec x + 3 \vec y - 1\vec w = 0$:
$2\cdot \begin{bmatrix}1 \\0 \end{bmatrix} + 3\cdot \begin{bmatrix}0 \\1 \end{bmatrix} - 1 \begin{bmatrix}2 \\3 \end{bmatrix} = \begin{bmatrix}0 \\0 \end{bmatrix}$
Intuitively, this is what is meant by the proof.  When these factors hold, it means that at least one vector is essentially "redundant" and unnecessary — it's just a combination of the other vectors. 
More generally, assume you have a set of vectors with this property:
$a_1 \vec v_1 + a_2  \vec v_2 + ... + a_n \vec v_n = 0 $
This can be written as:
$-a_1 \vec v_1 = a_2  \vec v_2 + ... + a_n  \vec v_n$
Which means that $v_1$ can be expressed as simply a linear combination of the other vectors:
$v_1 = a_2' v_2 + ... + a_n'v_n$
When you have a set of vectors like this, it means that you have multiple ways to express the same point using your coordinate system.  For example, with the vectors above, a point could be identified in the standard x/y coordinate system as (8, 15).  But the same point could also be expressed as (4, 3) in w/y coordinates or as (5, -2) in w/x coordinates:
$8\vec x + 15 \vec y = 8\begin{bmatrix}1 \\0 \end{bmatrix} + 15 \begin{bmatrix}0 \\1 \end{bmatrix} = \begin{bmatrix}8 \\15 \end{bmatrix}$
$4\vec w + 3 \vec y = 4\begin{bmatrix}2 \\3 \end{bmatrix} + 3\begin{bmatrix}0 \\1 \end{bmatrix} = \begin{bmatrix}8 \\15 \end{bmatrix}$
$5\vec w -2 \vec x = 5\begin{bmatrix}2 \\3 \end{bmatrix} -2\begin{bmatrix}1 \\0 \end{bmatrix} = \begin{bmatrix}8 \\15 \end{bmatrix}$
We'd normally rather have a minimal set of vectors so that there's only one way to represent each point.  It keeps the math simpler.  This is similar to a system of equations in which you have three equations for two unknowns.  In the above, since we have $\vec w = 2 \vec x + 3 \vec y$, we can always substitute $\vec x$ and $\vec y$ for $\vec w$.  
Note that there's no "correct" coordinate system -- any set of two vectors here, $\{\vec x, \vec y\}$, $\{\vec w, \vec y\}$, or $\{\vec w, \vec x\}$, would provide a valid basis for a two-dimensional coordinate system.  But there's no need to have three vectors here to represent a point in two-dimensional space.
A: 
How the formal definition works... I'm not sure how a linear combination constructed in a certain way can tell you the variables are independent or not.

Perhaps a more intutive definition of linear dependence is as follows:
A set of vectors is linearly dependent if one of its vectors is a linear combination of the other vectors.
In other words, one of its vectors "depends linearly" on the other vectors. This definition can be deduced from the formal definition:
If $\{ \mathbf{v}_1, \dots, \mathbf{v}_n \}$ is linearly dependent, then there exists some non-zero $\alpha_1, \dots, \alpha_n$ such that
$$ \alpha_1 \mathbf{v}_1 + \alpha_2 \mathbf{v}_2 + \dots + \alpha_n \mathbf{v}_n = \mathbf{0} $$
Without loss of generality, suppose $\alpha_1 \neq 0$ then we have
$$ \mathbf{v}_1 = - \frac{\alpha_2}{\alpha_1} \mathbf{v}_2 - \dots - \frac{\alpha_n}{\alpha_1} \mathbf{v}_n $$
So $\mathbf{v}_1$ is a linear combination of the other vectors.

Why set the linear combination equations to $\mathbf{0}$. I don't see how setting to zero helps determine independence or not.

In the proof above, we did not set the right-hand side to zero. It is zero only because everything is moved to the left-hand side.

Why choose $\alpha_k$ to be non-zero in one case. It seems arbitrary.

In the proof above, we need not work with $\alpha_1$. We could have chosen any non-zero $\alpha_k$ to show that $\mathbf{v}_k$ is a linear combination of the other vectors.
A: Your basic intuition is quite right --- a set of vectors are linearly independent if they don't affect each other (which, as it stands, is a somewhat ambiguous statement, hence the formal definition).
Perhaps it would be helpful to start with just two vectors, say the standard vectors
$$
\begin{pmatrix} 1\\ 0 \end{pmatrix} \hspace{20pt}\text{and}\hspace{20pt} \begin{pmatrix} 0\\ 1 \end{pmatrix}
$$
which are linearly independent in $\mathbb{R}^{2}$ (two-dimensional coordinate space). Natural intuition would be to think that these two are linearly independent because there is no scalar that you can multiply one by to get the other. That is, for $\alpha \in \mathbb{R}$, the equation
$$
\alpha\begin{pmatrix} 1\\ 0 \end{pmatrix} = \begin{pmatrix} 0\\ 1 \end{pmatrix}
$$
has no solutions. This is a good rule for any set of two vectors, and is in fact equivalent to the formal definition for if there were scalars $\alpha_{1}, \alpha_{2} \in \mathbb{R}$ such that
$$
\alpha_{1}\begin{pmatrix} 1\\ 0 \end{pmatrix} + \alpha_{2}\begin{pmatrix} 0\\ 1 \end{pmatrix} = 0
$$
(which is just the formal definition), then we could rearrange it to get
$$
-\frac{\alpha_{1}}{\alpha_{2}}\begin{pmatrix} 1\\ 0 \end{pmatrix} = \begin{pmatrix} 0\\ 1 \end{pmatrix}.
$$
This is clearly false!
The formal definition for an arbitrary number of vectors is an extension of this idea. We pick $0$ to be the right hand side for convenience, since every vector space must have the zero vector (by definition), so that if some vectors affect each other in some way, they must be able to be combined some way to give you the zero vector. In fact, if a set of vectors are linearly independent, then the only vector that they agree on is the zero vector, and it can only be obtained by multiplying each vector by zero (and summing them up).
If they can be combined in some way to give you the zero vector, then you can just rearrange them (like above) and find one of the vectors as a linear combination of the others, which would clearly imply that it is not independent of the others.
To make this more clear, let $\{v_{1}, v_{2}, \ldots, v_{n}\}$ be a set of vectors, and suppose there exists non-zero scalars $\alpha_{i} \in \mathbb{R}$ such that
$$
\alpha_{1}v_{1} + \alpha_{2}v_{2} + \ldots + \alpha_{n}v_{n} = 0.
$$
Then you can just rearrange this to get, for example, the equation
$$
\frac{-\alpha_{1}}{\alpha_{n}}v_{1} + \frac{-\alpha_{2}}{\alpha_{n}}v_{2} + \ldots + \frac{-\alpha_{n-1}}{\alpha_{n}}v_{n-1} = v_{n}.
$$
Then $v_{n}$ depends on the other vectors, so the set is not linearly independent! Thus, the only way that you can get a set of linearly independent vectors to all give you the zero vector, is that all must be multiplied by zero, else you can rearrange the equation to show that one of the vectors depends on the others, which would be a contradiction. This is why the definition is the way it is.
As a note for your third point, the definition wants $\alpha_{i} = 0$ for all $i = 1, \ldots, n$. That is, every coefficient has to be zero, not just one of them.
A: If we are given two nonparallel vectors $u,v$ in the space, they span a plane, a 2d subspace. Any vector on that plane can be expressed as $\lambda u+\mu v$, these will be the dependent vectors for $u,v$, everybody else will be independent from $u,v$. 
 
And, by the way, $\lambda,\mu$ are the coordinates w.r.t. basis $u,v$ of plane $\mathrm{span}(u,v)$. 

The geometrical meaning of $n$ vectors being linearly independent is that they span exactly $n$ dimensions, 
  alternatively, they form a basis for the subspace they generate. 


A main feature of vector spaces is that we can set up coordinate systems and coordinate all vectors in a unique way.
Setting up a coordinate system is equivalent to give basis vectors. 
A system of vectors is good for coordinating all vectors iff it is a generating system, i.e. generates the whole vector space (by means of linear combinations). 
A system of vectors $v_k$ is good for coordinating vectors in a unique way iff it is linearly independent.


*

*Indeed, if $\lambda_1v_1+\dots+\lambda_nv_n=\mu_1v_1+\dots+\mu_nv_n$ are two different coordinates for the same vectors, then $(\lambda_1-\mu_1)v_1+\dots+(\lambda_n-\mu_n)v_n=0$ with at least 1 nonzero coefficient.

*If, on the other hand, $v_i$ uniquely coordinates the vectors, in particular it does so for $0$. 

A: Your intuition may be leading you astray. You write "linear independence is where the variables are independent and don't affect each other." But vectors are not variables. $(1, 0)$, $(0, 1)$, and $(1, 1)$ are linearly dependent, but that's because you can get $(1, 1)$ by adding the first two vectors together. It's not because changing one of the vectors will change the other, since they are what they are. 
The idea of one vector being dependent on others is that you can get the first vector by scaling the other vectors appropriately and then adding the scaled vectors together. For instance, $(2, 4)$ is dependent on $(1, 0)$ and $(0, 1)$ because 
$$
2 \cdot (1, 0) + 4 \cdot (0, 1) = (2, 4)
$$
Since you can represent $(2, 4)$ by multiplying $(1,0)$ and $(0,1)$ by constants and adding the results together, it is not independent of them. 
But you could take the equation and move the $(2,4)$ to the other side, like this:
$$
2 \cdot (1, 0) + 4 \cdot (0, 1) + (-1) \cdot(2,4) = 0
$$
And that's an example of just what the definition is saying. When you say that you can get a sum like that to add up to zero, it's the same as saying that you can get any one of the vectors by multiplying the others by appropriate constants and adding them. And that's what it means to not be independent.
A: Each of $n$ vectors in the collection determines a line. If it is possible to find $n$ men of various strength that, when they pull or push the point along those lines, the point does not move, then the vectors are dependent. It is because one of those men can neutralize the collective effort of all others. In other words, it is possible to apply nonzero forces along the lines to balance the system. 
If the only possibility for the point to stay still is to apply zero forces along the lines then the vectors are independent, i.e. each direction cannot be affected/neutralized by others.

A: I hesitate to add yet another answer, but nowhere do I see the geometric interpretation clearly mentioned.
You know that two points determine a line and three points determine a plane. If higher dimensional geometry is allowed, this pattern continues: 4 points determine a 3-plane (3 dimensional hyperplane = a space), 5 points determine a 4-plane, etc.
But, of course, that is a synopsis that passes over a very important qualification: 3 points determine a plane only if they are not co-linear. 4 points determine a 3-plane only if they are not coplanar. So to be explicit, we define the concept of "general position": A collection of $n+1$ points is in general position if none of the points lies in the smallest linearly-closed set containing the remaining points ("linearly-closed" meaning that given any two distinct points in the set, the entire line through the points is also in the set). $n+1$ points determine a unique $n$-plane if and only if they are in general position. Otherwise they only determine a $k$-plane for some $k < n$.
This is exactly what is going on with linear dependence/independence. With vector spaces, we have one point given to us gratis: the origin. The vectors in the set whose dependence we are examining are the remaining geometric points. If a set of two vectors is linearly dependent, then one of the vectors lies on the line determined by the other vector and the origin (and vice versa), so the three points (origin and two vectors) are not in general position. Three vectors are linearly dependent if at least one of them lies in the plane (i.e., the span, which is the smallest vector subspace) determined by the other two and the origin, and are linearly independent if each vector does not lie in the span of the other two. Et cetera.
A: Imagine you have a collection of arrows pointing in various directions. If they're linearly dependent, then you can stretch, shrink, and reverse (but not rotate) them in such a way that if you lay them head-to-tail then they form a closed loop. For example, if you have three arrows that happen to all lie in the same plane (linearly dependent), then you can form a triangle out of them, but you can't if one of them sticks out of the plane formed by the other two (linearly independent).
A: Before we grapple with linear independence, it might be best to figure out what linear dependence means first.
When I think of the phrase "linear dependence" with regards to a set of vectors, what comes to mind for me is that, in some sense, one of those vectors "depends" on the other vectors in the set.  The mathematical formalization for this dependence is:

A set of nonzero vectors $\{\mathbf{v}_k \}_{k=1}^n$ in a vector space $V$ over a field $F$ is called linearly dependent when there exists$^\dagger$ a $j$ so that one can write $\displaystyle \mathbf{v}_j = \sum_{k \neq j} c_k \mathbf{v}_k$, where $c_k \in F$.

So we say $\mathbf{v}_j$ "depends" on the other vectors: for any $d \in \mathbb{R}$, one can arrive at the point $d\mathbf{v}_j$ simply by travelling some distance in each of the other directions$^\ddagger$ $\mathbf{v}_{k \neq j}$.  In other words, the span of the entire set of vectors is the same as the span of that same set of vectors but with $\mathbf{v}_j$ excluded. Note that, because $\mathbf{v}_j$ is nonzero, we must have $c_k \neq 0$ for at least one $k$.
Rewriting the above as $\mathbf{v}_j - \displaystyle \sum_{k \neq j} c_k \mathbf{v}_k = 0$, we can now deduce that a set of $n$ vectors is linearly dependent $\iff$ we can find a set of constants $\{c_k\}_{k=1}^n$, not all zero, so that $\displaystyle \sum_{k=1}^n c_k \mathbf{v}_k = 0$.  This statement is equivalent to our definition for linear independence:

A set of vectors $\{\mathbf{v}_k \}$ is called linearly independent $\iff$ we cannot find a set of constants $\{c_k\}_{k=1}^n$, not all zero, so that $\displaystyle \sum_{k=1}^n c_k \mathbf{v}_k = 0$.  This is to say, $c_k = 0$ for all $k$ is the only solution to this equation.


$^\dagger$ This $j$ is never unique.

$^\ddagger$ To give a concrete example, let  $\displaystyle \mathbf{v}_1 = \left[1 \atop 0 \right],  \mathbf{v}_2 = \left[0 \atop 1\right]$, and $\displaystyle \mathbf{v}_3 = \left[1 \atop 1\right]$ in $\mathbb{R}^2$.  See that $\mathbf{v}_3$ depends on $\mathbf{v}_1$ and $\mathbf{v}_2$ since you can get to the point $d \mathbf{v}_3$ for any $d \in \mathbb{R}$ by first travelling $d$ units in the $\mathbf{v}_1$ direction and then $d$ units in the $\mathbf{v}_2$ direction.
A: You have linear dependence when a vector can be expressed as a linear combination of the others in the set. Dependence in this sense is like the $y$ in $y=f(x)$, meaning that the $y$ depends on the values of $x$ and is determined by it through $f$. This happens when you can write a linear combination of the vectors that equals to zero, since you can move one of the vectors on the other side and divide by $-a_{i}$. Of course the coefficient must not be all zeroes otherwise you would just have $0=0$.
A: For the intuition of the general case let consider the simplest case $V=\mathbb{R^2}$ and the two sets


*

*$S=\{(1,0),(1,1),(0,1)\}$

*$T=\{(1,0),(1,1)\}$
and apply the definitions, that is for $S$


*

*$1\cdot (1,0)-1\cdot (1,1)+1\cdot (0,1)=(1-1,-1+1)=(0,0)$


and for $T$


*

*$c_1(1,0)+c_2(1,1)=(c_1+c_2,c_2)=(0,0) \implies c_1=c_2=0$

