I'm currently taking a university class on linear algebra. In some proofs, a given matrix $A \in \mathbb{R}^{m\times n}$ is said to be able to be represented as a $1\times n$ row vector of $m \times 1$ column vectors, i.e.: $$ A = [\vec{a_1}\quad\vec{a_2}\quad\dots\quad \vec{a_n}] $$

with $\vec{a_i}$ the ith column of $A$. Naturally, the transpose of $A$, as used in most such proofs, would then be given by an $n\times 1$ column vector of $1\times m$ row vectors: $$ A^T = [\vec{a_1^T}\quad\vec{a_2^T}\quad\dots\quad \vec{a_n^T}]^T $$ When performing matrix-vector multiplication of the form $A\vec x$, I know the vector's amount of rows has to match the matrix's amount of columns. (In this case, $\vec x$ would be an $n \times 1$ column vector, and the result would be an $[m \times n][n \times 1] = [m \times 1]$ column vector.)

Now, when considering $A$ as a row vector as in the first equation, we can see that this holds up, as the result would be $[1 \times n][n \times 1] = [1 \times 1]$, though containing a sum of scaled $m \times 1$ column vectors, so, after scaling and adding, an $m \times 1$ column vector. $$ A\vec x = [\vec{a_1}\quad\vec{a_2}\quad\dots\quad \vec{a_n}][x_1\quad x_2 \quad\dots\quad x_n]^T = \vec{a_1}\cdot x_1 + \vec{a_2}\cdot x_2 + \dots + \vec{a_n}\cdot x_n \in \mathbb{R}^{m \times 1} $$ My problem, then, arises when considering the same product, but swapping $A$ with $A^T$ (Edit: For clarity, this is not transposing $Ax$, but rather interchanging $A$ to see what happens.), thus $A^T\vec x$ with $\vec x$ the same $n \times 1$ column vector. Conventionally, this would be the product of an $n \times m$ matrix with an $n \times 1$ vector: impossible. However, when considering $A^T$ as an $n \times 1$ column vector like in the second equation, this does seem to become possible, namely as the dot of two vectors of equal dimensions: $$ A^T\boldsymbol{\cdot}\vec x = [\vec{a_1^T}\quad\vec{a_2^T}\quad\dots\quad \vec{a_n^T}]^T\boldsymbol{\cdot}[x_1\quad x_2 \quad\dots\quad x_n]^T = \vec{a_1^T}\cdot x_1 + \vec{a^T_2}\cdot x_2 + \dots + \vec{a^T_n}\cdot x_n \in \mathbb{R}^{1\times n} $$ This, to me, seems to be a paradox, as there obviously is a mismatch in the dimensions of $\vec x$ (i.e. $n$) and the dimensions of the input space of $A^T$ (i.e. $m$), yet, rewriting $A^T$ as a vector, as is done with $A$, eliminates said mismatch, as it seems. Is it wrong to assume one can eliminate this mismatch? Is it wrong to assume the matrix-vector product of a matrix in $\mathbb{R}^{n\times 1}$ with a vector in $\mathbb{R}^n$ to be equal to the dot product of two vectors in $\mathbb{R}^n$? Is it wrong to write a matrix as a vector of vectors, or perhaps to transpose such a construction?

  • $\begingroup$ If you transpose $Ax$ you get $x^T A^T$, not $A^T x$, which generally won't make sense. The product you write down in your last equation doesn't make sense either. $\endgroup$ Nov 14 '18 at 9:35
  • $\begingroup$ Yes, I know $(AB)^T = B^TA^T$, but that's not what I'm asking about. $\endgroup$
    – Mew
    Nov 14 '18 at 9:44

There are 2 wrong things:

1) Although you can write an $n \times m$ matrix as a table with $m $ columns each of which is a column vector of $n $ entries, this does not mean you are allowed to consider that its dimension is $1 \times m $. It is still $n \times m$.

2) the dot product of vectors $a $ and $b $ is carried out as $a^T \, b $ so dimensions must not be equal, the must be $1 \times n $ and $n \times 1$.

  • $\begingroup$ I guess the first remark solves my issue; i.e., a "vector of vectors" is a slippery term that does not behave like a regular vector, even though it is used fairly often to explain matrix multiplication (among other topics like orthogonal projection). The second remark is probably a misunderstanding of what I wrote, or perhaps of my use of transposes for $\vec x$ as to not clutter the page with columns: when dotting $\vec a$ and $\vec b$ as $\vec a\cdot \vec b$, both vectors ought to be of identical dimensions, but of course, representing it as a matrix-vector product transposes $\vec a$. $\endgroup$
    – Mew
    Nov 18 '18 at 22:06
  • $\begingroup$ I do believe that the second point holds, because in your question, when you interchange $A$ with $A^T$, the resulting expression $A^T \vec{x}$ is a "matrix-vector product" as you call it, not a dot product, so dimensions really must not be identical. $\endgroup$
    – Javi
    Nov 19 '18 at 22:51
  • $\begingroup$ Ah, that's a notational error on my part, I see what you mean now. The intended expression was $A^T\boldsymbol{\cdot}\vec{x}$. $\endgroup$
    – Mew
    Nov 22 '18 at 16:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.