I'm currently taking a university class on linear algebra. In some proofs, a given matrix $A \in \mathbb{R}^{m\times n}$ is said to be able to be represented as a $1\times n$ row vector of $m \times 1$ column vectors, i.e.: $$ A = [\vec{a_1}\quad\vec{a_2}\quad\dots\quad \vec{a_n}] $$

with $\vec{a_i}$ the ith column of $A$. Naturally, the transpose of $A$, as used in most such proofs, would then be given by an $n\times 1$ column vector of $1\times m$ row vectors: $$ A^T = [\vec{a_1^T}\quad\vec{a_2^T}\quad\dots\quad \vec{a_n^T}]^T $$ When performing matrix-vector multiplication of the form $A\vec x$, I know the vector's amount of rows has to match the matrix's amount of columns. (In this case, $\vec x$ would be an $n \times 1$ column vector, and the result would be an $[m \times n][n \times 1] = [m \times 1]$ column vector.)

Now, when considering $A$ as a row vector as in the first equation, we can see that this holds up, as the result would be $[1 \times n][n \times 1] = [1 \times 1]$, though containing a sum of scaled $m \times 1$ column vectors, so, after scaling and adding, an $m \times 1$ column vector. $$ A\vec x = [\vec{a_1}\quad\vec{a_2}\quad\dots\quad \vec{a_n}][x_1\quad x_2 \quad\dots\quad x_n]^T = \vec{a_1}\cdot x_1 + \vec{a_2}\cdot x_2 + \dots + \vec{a_n}\cdot x_n \in \mathbb{R}^{m \times 1} $$ My problem, then, arises when considering the same product, but swapping $A$ with $A^T$ (Edit: For clarity, this is not transposing $Ax$, but rather interchanging $A$ to see what happens.), thus $A^T\vec x$ with $\vec x$ the same $n \times 1$ column vector. Conventionally, this would be the product of an $n \times m$ matrix with an $n \times 1$ vector: impossible. However, when considering $A^T$ as an $n \times 1$ column vector like in the second equation, this does seem to become possible, namely as the dot of two vectors of equal dimensions: $$ A^T\boldsymbol{\cdot}\vec x = [\vec{a_1^T}\quad\vec{a_2^T}\quad\dots\quad \vec{a_n^T}]^T\boldsymbol{\cdot}[x_1\quad x_2 \quad\dots\quad x_n]^T = \vec{a_1^T}\cdot x_1 + \vec{a^T_2}\cdot x_2 + \dots + \vec{a^T_n}\cdot x_n \in \mathbb{R}^{1\times n} $$ This, to me, seems to be a paradox, as there obviously is a mismatch in the dimensions of $\vec x$ (i.e. $n$) and the dimensions of the input space of $A^T$ (i.e. $m$), yet, rewriting $A^T$ as a vector, as is done with $A$, eliminates said mismatch, as it seems. Is it wrong to assume one can eliminate this mismatch? Is it wrong to assume the matrix-vector product of a matrix in $\mathbb{R}^{n\times 1}$ with a vector in $\mathbb{R}^n$ to be equal to the dot product of two vectors in $\mathbb{R}^n$? Is it wrong to write a matrix as a vector of vectors, or perhaps to transpose such a construction?

  • $\begingroup$ If you transpose $Ax$ you get $x^T A^T$, not $A^T x$, which generally won't make sense. The product you write down in your last equation doesn't make sense either. $\endgroup$ Commented Nov 14, 2018 at 9:35
  • $\begingroup$ Yes, I know $(AB)^T = B^TA^T$, but that's not what I'm asking about. $\endgroup$
    – Mew
    Commented Nov 14, 2018 at 9:44

1 Answer 1


There are 2 wrong things:

1) Although you can write an $n \times m$ matrix as a table with $m $ columns each of which is a column vector of $n $ entries, this does not mean you are allowed to consider that its dimension is $1 \times m $. It is still $n \times m$.

2) the dot product of vectors $a $ and $b $ is carried out as $a^T \, b $ so dimensions must not be equal, the must be $1 \times n $ and $n \times 1$.

  • $\begingroup$ I guess the first remark solves my issue; i.e., a "vector of vectors" is a slippery term that does not behave like a regular vector, even though it is used fairly often to explain matrix multiplication (among other topics like orthogonal projection). The second remark is probably a misunderstanding of what I wrote, or perhaps of my use of transposes for $\vec x$ as to not clutter the page with columns: when dotting $\vec a$ and $\vec b$ as $\vec a\cdot \vec b$, both vectors ought to be of identical dimensions, but of course, representing it as a matrix-vector product transposes $\vec a$. $\endgroup$
    – Mew
    Commented Nov 18, 2018 at 22:06
  • $\begingroup$ I do believe that the second point holds, because in your question, when you interchange $A$ with $A^T$, the resulting expression $A^T \vec{x}$ is a "matrix-vector product" as you call it, not a dot product, so dimensions really must not be identical. $\endgroup$
    – Javi
    Commented Nov 19, 2018 at 22:51
  • $\begingroup$ Ah, that's a notational error on my part, I see what you mean now. The intended expression was $A^T\boldsymbol{\cdot}\vec{x}$. $\endgroup$
    – Mew
    Commented Nov 22, 2018 at 16:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .