# Matrix-vector product: representing matrix as vector of vectors seemingly leads to paradox when transposing the matrix

I'm currently taking a university class on linear algebra. In some proofs, a given matrix $$A \in \mathbb{R}^{m\times n}$$ is said to be able to be represented as a $$1\times n$$ row vector of $$m \times 1$$ column vectors, i.e.: $$A = [\vec{a_1}\quad\vec{a_2}\quad\dots\quad \vec{a_n}]$$

with $$\vec{a_i}$$ the ith column of $$A$$. Naturally, the transpose of $$A$$, as used in most such proofs, would then be given by an $$n\times 1$$ column vector of $$1\times m$$ row vectors: $$A^T = [\vec{a_1^T}\quad\vec{a_2^T}\quad\dots\quad \vec{a_n^T}]^T$$ When performing matrix-vector multiplication of the form $$A\vec x$$, I know the vector's amount of rows has to match the matrix's amount of columns. (In this case, $$\vec x$$ would be an $$n \times 1$$ column vector, and the result would be an $$[m \times n][n \times 1] = [m \times 1]$$ column vector.)

Now, when considering $$A$$ as a row vector as in the first equation, we can see that this holds up, as the result would be $$[1 \times n][n \times 1] = [1 \times 1]$$, though containing a sum of scaled $$m \times 1$$ column vectors, so, after scaling and adding, an $$m \times 1$$ column vector. $$A\vec x = [\vec{a_1}\quad\vec{a_2}\quad\dots\quad \vec{a_n}][x_1\quad x_2 \quad\dots\quad x_n]^T = \vec{a_1}\cdot x_1 + \vec{a_2}\cdot x_2 + \dots + \vec{a_n}\cdot x_n \in \mathbb{R}^{m \times 1}$$ My problem, then, arises when considering the same product, but swapping $$A$$ with $$A^T$$ (Edit: For clarity, this is not transposing $$Ax$$, but rather interchanging $$A$$ to see what happens.), thus $$A^T\vec x$$ with $$\vec x$$ the same $$n \times 1$$ column vector. Conventionally, this would be the product of an $$n \times m$$ matrix with an $$n \times 1$$ vector: impossible. However, when considering $$A^T$$ as an $$n \times 1$$ column vector like in the second equation, this does seem to become possible, namely as the dot of two vectors of equal dimensions: $$A^T\boldsymbol{\cdot}\vec x = [\vec{a_1^T}\quad\vec{a_2^T}\quad\dots\quad \vec{a_n^T}]^T\boldsymbol{\cdot}[x_1\quad x_2 \quad\dots\quad x_n]^T = \vec{a_1^T}\cdot x_1 + \vec{a^T_2}\cdot x_2 + \dots + \vec{a^T_n}\cdot x_n \in \mathbb{R}^{1\times n}$$ This, to me, seems to be a paradox, as there obviously is a mismatch in the dimensions of $$\vec x$$ (i.e. $$n$$) and the dimensions of the input space of $$A^T$$ (i.e. $$m$$), yet, rewriting $$A^T$$ as a vector, as is done with $$A$$, eliminates said mismatch, as it seems. Is it wrong to assume one can eliminate this mismatch? Is it wrong to assume the matrix-vector product of a matrix in $$\mathbb{R}^{n\times 1}$$ with a vector in $$\mathbb{R}^n$$ to be equal to the dot product of two vectors in $$\mathbb{R}^n$$? Is it wrong to write a matrix as a vector of vectors, or perhaps to transpose such a construction?

• If you transpose $Ax$ you get $x^T A^T$, not $A^T x$, which generally won't make sense. The product you write down in your last equation doesn't make sense either. – Qiaochu Yuan Nov 14 '18 at 9:35
• Yes, I know $(AB)^T = B^TA^T$, but that's not what I'm asking about. – Mew Nov 14 '18 at 9:44

1) Although you can write an $$n \times m$$ matrix as a table with $$m$$ columns each of which is a column vector of $$n$$ entries, this does not mean you are allowed to consider that its dimension is $$1 \times m$$. It is still $$n \times m$$.
2) the dot product of vectors $$a$$ and $$b$$ is carried out as $$a^T \, b$$ so dimensions must not be equal, the must be $$1 \times n$$ and $$n \times 1$$.
• I guess the first remark solves my issue; i.e., a "vector of vectors" is a slippery term that does not behave like a regular vector, even though it is used fairly often to explain matrix multiplication (among other topics like orthogonal projection). The second remark is probably a misunderstanding of what I wrote, or perhaps of my use of transposes for $\vec x$ as to not clutter the page with columns: when dotting $\vec a$ and $\vec b$ as $\vec a\cdot \vec b$, both vectors ought to be of identical dimensions, but of course, representing it as a matrix-vector product transposes $\vec a$. – Mew Nov 18 '18 at 22:06
• I do believe that the second point holds, because in your question, when you interchange $A$ with $A^T$, the resulting expression $A^T \vec{x}$ is a "matrix-vector product" as you call it, not a dot product, so dimensions really must not be identical. – Javi Nov 19 '18 at 22:51
• Ah, that's a notational error on my part, I see what you mean now. The intended expression was $A^T\boldsymbol{\cdot}\vec{x}$. – Mew Nov 22 '18 at 16:39