So far I have understood that a set of vectors $S = {v_1, v_2, . . . , v_k }$ in a vector space V is linearly independent when the vector equation $c_1v_1 + c_2v_2 + . . . + c_kv_k = 0$ has only the trivial solution$c_1 = 0, c_2 = 0, . . . , c_k = 0.$

An example in matrix form is:

$\begin{bmatrix}1 & 1 & 2 & 4 \\ 0 & -1 & -5 & 2 \\ 0 & 0 & -4 & 1 \\ 0 & 0 & 0 & 6 \\ \end{bmatrix} \begin{bmatrix} c_1\\ c_2 \\ c_3 \\ c_4 \\ \end{bmatrix} = \begin{bmatrix} 0\\ 0 \\ 0 \\ 0 \\ \end{bmatrix} $

But a matrix of this form

$\begin{bmatrix}1 & 1 & 2 & 4 \\ 0 & -1 & -5 & 2 \\ 0 & 0 & -4 & 1 \\ 0 & 0 & 0 & 0 \\ \end{bmatrix} \begin{bmatrix} c_1\\ c_2 \\ c_3 \\ c_4 \\ \end{bmatrix} = \begin{bmatrix} 0\\ 0 \\ 0 \\ 0 \\ \end{bmatrix} $

is linearly dependent because it has more than a trivial solution

However, I am confused about row vectors, specifically the idea that to get a basis for a subspace using row vectors we must put the matrix in reduced row echelon form to find the linearly independent vectors. For example here the accepted answer gives an example of finding a basis with row vectors using this

$\begin{bmatrix}1 & 1 & 2 & 4 \\ 2 & -1 & -5 & 2 \\ 1 & -1 & -4 & 0 \\ 2 & 1 & 1 & 6 \\ \end{bmatrix} \Rightarrow \begin{bmatrix}1 & 1 & 2 & 4 \\ 0 & -3 & -9 & -6 \\ 0 & -2 & -6 & -4 \\ 0 & -1 & -3 & -2 \\ \end{bmatrix} \Rightarrow \begin{bmatrix}1 & 1 & 2 & 4 \\ 0 & -3 & -9 & -6 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \\ \end{bmatrix}$

and then goes on to say that "Only two of the four original vectors were linearly independent." In what respect are these two vectors linearly independent? This looks exactly like the second example that I gave in which the vectors were dependent because they had more than a trivial solution? Does linear independence with regard to row vectors mean something else? Or does this also only have a trivial solution, and if so, how?

  • $\begingroup$ Row rank = column rank. $\endgroup$ – Wuestenfux Mar 26 at 12:23
  • 2
    $\begingroup$ Would you care to explain a little more? I am familiar with what rank is, but I am not sure how it answers my question. $\endgroup$ – agblt Mar 26 at 12:28
  • $\begingroup$ In your second example you concluded that there are more than one solution, but the number of non-zero echelons tells you more. If you take the number of unknowns (number of columns) and you subtract the number of non-zero echelons, that tells you the dimension of the set of solutions. In your example that would be $4-3=1$. Therefore, the solutions of that system form a line. In the last example it would be $4-2=2$. Therefore, the set of solutions forms a plane. The number of linearly independent rows tells you how many dimensions the system is cutting down from the total space. $\endgroup$ – user647486 Mar 26 at 12:39

Linear independence is linear independence is linear independence. It's defined entirely independently of matrices. It is, instead, defined in terms of a vector equation, which in finite dimensions, can be turned into a system of linear equations. As such, matrices are an excellent tool to determine linear independence.

The definition of linear independence is precisely what you wrote. We say $v_1, \ldots, v_n$ are linearly independent if the only solution of $$a_1 v_1 + \ldots + a_n v_n = 0 \tag{$\star$}$$ for scalars $a_1, \ldots, a_n$, is the trivial solution $a_1 = a_2 = \ldots = a_n = 0$. That is, no other possible choices of scalars will make the above linear combination into the $0$ vector.

This doesn't matter if they are row vectors, column vectors, or more abstract vectors (such as matrices, functions, graphs on a fixed vertex set, algebraic numbers, etc). All you need to define linear independence is an (abstract) vector space.

For example, the real functions $\sin^2(x)$, $\cos^2(x)$, and the constant function $1$ are not linearly independent because $$2 \cdot \sin^2(x) + 2 \cdot \cos^2(x) - 2 \cdot 1 \equiv 0,$$ i.e. the linear combination is exactly the $0$ function, even though the scalars aren't all $0$.

On the other hand, the functions $\sin^2$ and $\cos^2$ are linearly independent, because, if we assume $$a_1 \sin^2(x) + a_2 \cos^2(x) \equiv 0,$$ that is, is equal to $0$ for all $x$, then trying $x = 0$ yields $$0 = a_1 \sin^2(0) + a_2 \cos^2(0) = a_2$$ and trying $x = \pi/2$ yields $$0 = a_1 \sin^2(\pi/2) + a_2 \cos^2(\pi/2) = a_1.$$ Thus, we logically come to the conclusion that $a_1 = a_2 = 0$, i.e. the functions are linearly independent.

So, where do matrices come in? If our vectors belong to $\Bbb{R}^m$ (or $\Bbb{C}^m$, or indeed $\Bbb{F}^m$ where $\Bbb{F}$ is a field), then equation $(\star)$ turns into a system of homogeneous linear equations. When you turn this system of linear equations into a matrix of coefficients, the columns will turn out to be precisely the vectors $v_1, \ldots, v_n$, expressed as column vectors. It doesn't matter whether $v_1, \ldots, v_n$ are expressed originally as column vectors or row vectors! Once you turn them into equations, then a matrix, they will become columns. (You should try this for yourself to convince yourself of this fact.)

So, if you take the rows of a given matrix, and try to figure out (by definition) whether they are linearly independent or not, you'll inevitably end up with these vectors being columns, i.e. you'll get the same matrix, just transposed.

Further, we also get a nice technique for proving linear (in)dependence of vectors in $\Bbb{R}^3$, and pruning them down to a linearly independent set: stick them as columns in a matrix $A$, row reduce to a row-echelon form $B$, and if the $i$th column of $B$ does not have a pivot in it, then the $i$th column of $A$ depends linearly on the previous columns of $A$, and hence can be removed without damaging the span.

If you instead stick the vectors in as rows in a matrix and reduce as above, then this will not tell you which vectors depend on each other, in the same way that the column approach does. However, row operations preserve the span of the row vectors, hence the non-zero rows of a row-echelon form of a matrix will be a basis for the span of your vectors. This basis may have no vectors in common with your original set of vectors, however!

  • $\begingroup$ Thank you for the thorough explanation. What do you mean by "So, if you take the rows of a given matrix, and try to figure out (by definition) whether they are linearly independent or not, you'll inevitably end up with these vectors being columns, i.e. you'll get the same matrix, just transposed." As far as I can tell, in the example I brought from the link, we are treating the row vectors as rows in the matrix, i.e. we are doing row operations to them to get them to echelon form. Or am I misreading? $\endgroup$ – agblt Mar 26 at 13:34
  • $\begingroup$ e.g. take the matrix $\begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}$. Test linear independence of $(1, 2)$ and $(2, 3)$ by solving $a(1, 2) + b(3, 4) = (0, 0)$. Simplifying the left side gives us $(a + 3b, 2a + 4b) = (0, 0)$, or as a system of linear equations, $a + 3b = 0$ and $2a + 4b = 0$. If we put these equations into a coefficient matrix (to solve for $a$ and $b$), we get $\begin{pmatrix} 1 & 3 \\ 2 & 4 \end{pmatrix}$; naturally the transpose of what we started with! $\endgroup$ – Theo Bendit Mar 26 at 13:39
  • $\begingroup$ I think I almost understand now. Linear independence means exactly what it sounds like, vectors of a set that cannot be expressed as a linear combination of the other vectors. When vectors are in row form, this is found by performing operations on the rows until we have reduced the matrix to vectors that cannot be expressed as linear combinations of each other. When vectors are in column form, since column operations are not allowed, it is found by adding the linear combination of vectors to $0$ and seeing that no vector equals a linear combination of the others, if each column has a pivot. $\endgroup$ – agblt Mar 26 at 15:07
  • $\begingroup$ ...However the number of vectors that are independent of each other and form the basis of a subset are the same whether the vectors are in row or column form. That is why the row rank = column rank. Please correct me if I am getting something wrong! $\endgroup$ – agblt Mar 26 at 15:15
  • $\begingroup$ I'm finding it a little hard to follow your reasoning. I think I agree with the assessment about the rows, but I'm dubious about the columns stuff. If you want to show row and column ranks are equal, then show they are both the number of pivots. Because the non-zero rows of a RREF matrix all have pivots, the row rank is the number of pivots. Because a lack of a pivot in a column of a RREF matrix indicates that the original column was dependent on previous columns, it means that the number of pivots is the number of independent columns, i.e. the column rank. $\endgroup$ – Theo Bendit Mar 27 at 2:24

Yes, it is "exactly like the second example" you gave. In that example, you were showing that the original four vectors were dependent. But there will always be some subset of those vectors that is independent. (That subset might consist of a single vector. A set containing just one vector is always "independent".) In this last problem you are looking for that independent subset.

Your "four vectors" are the rows of the matrix (1, 1, 2, 4), (2, -1, -5, 2), (1, -1, -4, 0), and (2, 1, 1, 6). To determine whether they are "independent" or "dependent", you look for numbers, a, b, c, d, such that a(1, 1, 2, 4)+ b(2, -1, -5, 2)+ c(1, -1, -4, 0)+ d(2, 1, 1, 6)= (a+ 2b+ c+ 2d, a- b- c+ d, 2a- 5b- 4c+ d, 4a+ 2b+ 6d)= (0, 0, 0, 0). So we have the four equations, a+ 2b+ c+ 2d= 0, a- b- c+ d= 0, 2a- 5b- 4c+ d= 0, and 4a+ 2b+ 6d= 0. Adding the first two equations eliminates c: 2a+ b+ 3d= 0. But notice that the fourth equation, 4a+ 2b+ 6d= 0, is just two times that so is not really a new equation. If we add the third equation to 4 times the first equation we also eliminate c: 6a+ 3b+ 9d= 0. We can divide by 3 to get 2a+ b+ 3d= 0.

We have two equations, 3a+ b+ 3d= 0 and 2a+ b+ 3d= 0. Subtracting the second of those from the first gives a= 0. But then we have b+ 3d= 0 so b= -3d but d can be anything. Putting a= 0, b= -3d into a+ 2b+ c+ 2d= 0 we have -6d+ c+ 2d= c- 4d= 0 so c= 4d. That means that a(1, 1, 2, 4)+ b(2, -1, -5, 2)+ c(1, -1, -4, 0)+ d(2, 1, 1, 6)= (0, 0, 0, 0) can be written -3d(2, -1, -5, 2)+ 4d(1, -1, -4, 0)+ d(2, 1, 1, 1, 0)= (0, 0, 0, 0). Then d(2, 1, 1, 1 0)= 3d(2, -1, -5, 2)- 4d(1, -1, -4, 0) or, dividing by d, (2, 1, 1, 0)= 3(2, -1, -5, 2)- 4(1, -1, -4, 0). So those 4 vectors can actually be written in terms of these last two. These two vectors form an independent subset of the original four vectors. The space they span is actually two dimensional, not four dimensional. Using "row reduction" with a matrix is just a more convenient way of doing that.

  • $\begingroup$ $\{0\}$ is not linearly independent. The null set is always independent though. $\endgroup$ – jgon Mar 26 at 13:34
  • $\begingroup$ "We have two equations, 3a+ b+ 3d= 0 ...." Where did this one come from? $\endgroup$ – agblt Mar 26 at 14:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.