# What is the difference, geometrically, between row vectors and column vectors?

I have been told in the textbook I'm following that "By convention, always assume that vectors are in column orientation, unless stated otherwise." (Cohen, 2021: 27). Fair enough. But as I progress into the study of vector spaces,this distinction between row and column vectors seems to take on more significance.

In particular,I am introduced to fields and vector spaces and being told that "The space $$\mathbb{R}^n$$ consists of all column vectors with $$n$$ components". So is this different from having your space made of all row vectors with n components?

Generally: what is the geometrical interpretation of the distinction between row vectors and column vectors? (As I understand it, algebraically, vectors are just ordered list of numbers. The different between row vectors and column vectors here only shows up in the outcome of basic vectors operations like the dot product.)

• ~~ everything is dual ~~ Jul 27, 2022 at 16:22
• I think the author just wants you to get used to writing the components of a vector in a column in order to simplify the presentation of linear maps as matrices where the matrix is placed on the left of the column vector. The row vector is simply the transpose of a column vector. Jul 27, 2022 at 16:57
• Notice that there are many similar questions on this site, some of which are listed under the heading "Related" on this page. Are any of them helpful? If your question is uniquely different, it would be useful if you could explain more specifically what makes it unique. Jul 27, 2022 at 17:00
• To clarify my comment. It doesn't matter one jot from a pure perspective. However, the overwhelming convention is to use column vectors, so if we want to do dot products and related things sometimes we need to write vectors as row vectors to make the notation work seamlessly. That's all. You could as well write them diagonally up and diagonally down - notationally horrible, but, the pure mathematics of it is unchanged Jul 27, 2022 at 17:54
• @FShrike: Yes, my Markovian processes text used row vectors for discrete probability distributions, and post-multiplied them by the transition probability matrix. I found that mildly disorienting. Jul 27, 2022 at 17:55

It is important to not lose sight of what the component representations mean. We start with a vector space $$V$$. If $$V$$ is finite dimensional, we can take an ordered basis $$(e_i)_{i = 1}^n$$. For any vector $$v \in V$$ we may then uniquely write $$v = \sum_i v^i e_i$$. We say that the $$v^i$$ are the components of $$v$$ in the basis $$(e_i)$$. We traditionally represent the $$v^i$$ as a column vector $$\begin{bmatrix} v^1 \\ \vdots \\ v^n \end{bmatrix}$$ Notice that I say we traditionally do this. It is of course arbitrary how we decide to represent the components of $$v$$. We could have done it with a row vector or whatever else. But now suppose we want to talk about the components of a linear transformation $$A : V \to V$$. We can uniquely write $$Ae_i = \sum_j A_i^j e_j$$ We call the $$A_i^j$$ the components of $$A$$. Let's look at what happens when we act on $$v$$ with $$A$$. We calculate \begin{align} Av &= A \left( \sum_i v^i e_i \right) = \sum_i v^i Ae_i \\&= \sum_i v^i \left( \sum_j A_i^j e_j \right) \\&= \sum_j \left(\sum_i A_i^j v^i \right) e_j \end{align} We read off from this that the components of $$Av$$ are $$\sum_i A_i^j v^i$$. This is the usual rule for matrix multiplication.

Note that I have positioned indices so that we only ever sum over lower and upper pairs of indices. This might give us the inspired choice to represent matrices in $$2$$ dimensions. We'll say that upper indices run top to bottom in columns and lower indices run left to right in rows. So we represent $$A$$ with $$\begin{bmatrix} A_1^1 & \ldots & A_n^1 \\ \vdots & \ddots &\vdots \\ A_1^n & \ldots & A_n^n \end{bmatrix}$$ We see that insisting on this slightly (although not really as you progress in your mathematical career) unusual notation has the benefit of automatically reproducing the "row by column" matrix multiplication rule.

Now what kind of a thing is a row vector? Well really we shouldn't call it a row "vector" at all. It is, as we shall see, more properly called a row co-vector. A co-vector is a linear map $$f : V \to \mathbb{F}$$, where $$\mathbb{F}$$ is the underlying field for the vector space. Let's see if we can decide whether a co-vector should have upper or lower indices. We calculate $$f(v) = f\left(\sum_i v^i e_i \right) = \sum_i v^i f(e_i)$$ We see that $$f$$ is determined by the values $$f(e_i) \in \mathbb{F}$$. These values occur in a sum with an upper index. It stands to reason that we should say the components of $$f$$ are $$f_i = f(e_i)$$ with a lower index. According to our convention, then, we should represent $$f$$ as $$\begin{bmatrix} f_1 & \ldots & f_n \end{bmatrix}$$ We see that co-vectors are what we previously called row vectors. Indeed $$f(v) = \begin{bmatrix} f_1 & \ldots & f_n \end{bmatrix} \begin{bmatrix} v^1 \\ \vdots \\ v^n \end{bmatrix}$$ which is again consistent with the "row by column" multiplication rule.

Hopefully you can see that these are two very different objects. A column vector is a true vector, whereas a row vector is something that eats a vector and spits out a scalar.

But maybe you think we can just "flip" a row vector to obtain a vector. Of course we could, but then we would lose all the lovely and illuminating structure we've developed so far. We would just be forgetting the distinction between vectors and linear functions of vectors, which I hope you'll agree could be a worthwhile distinction.

If you're curious, a metric is the secret ingredient that lets us translate between row and column vectors. We often tacitly assume a metric that makes this translation the trivial "flip," which leads to a lot of confusion. So I emphasize again: row vectors and column vectors are naturally distinct. A column vector is a vector. A row vector is a co-vector.

• Is it that the components of the vector are the projection on the basis vector, while covector components are obtained by vector addition decomposition, the same for an orthogonal basis, or is it the other way round ? Jul 27, 2022 at 19:05
• The notion of a projection doesn't make sense until we have a metric. In brief, being able to swap between rows and columns is equivalent to having a notion of length in your vector space and we get both of these from a metric. A projection is the closest (notice we need a notion of length here) vector to another vector in some subspace. Jul 27, 2022 at 20:37
• Now you might ask why we ever do linear algebra without a metric. Let me give an example. You have $n$ goods you want to keep inventory on. A vector is $n$ real numbers saying how much you have of each. A co-vector could be the function that gives you the cost of a given vector (why is this linear?). We see that vectors and co-vectors really are different things here. Moreover, how long should the vector "1 apple and 2 oranges" be? Jul 27, 2022 at 20:45
• At this point you may be curious about metrics and vector spaces that are equipped with them. I encourage you to make a dedicated study of it. But for now it's important that you understand that once we have declared that column vectors are the true vectors in our vector space, a row vector is something fundamentally different. The row vectors are not, as we say, canonically isomorphic to the column vectors. This is a fact seldom emphasized in linear algebra courses and something of a hobby horse of mine. Jul 27, 2022 at 20:48
• There really is too much to say about all of this for a math stack exchange post, but if you like I can delete this answer and give a more pedagogically sound one. I was a bit rushed when I posted the first answer. Jul 27, 2022 at 21:02