# What is the “default” direction of feature / parameter vector?

This seems like a pretty obvious thing, so it's never really explained, but I can't understand it.

Many book chapters use the expression: $\boldsymbol w^{T}\boldsymbol x$ as a form of denoting something like $w_{1}x_{1} + w_{2}x_{2} + ... + w_{n}x_{n}$.

However if I treat $\boldsymbol w$ as a vector $\boldsymbol w = \begin{bmatrix} w_{1} & w_{2} & ... & w_{n} \end{bmatrix}$ and $\boldsymbol x$ as a vector $\boldsymbol x = \begin{bmatrix} x_{1} & x_{2} & ... & x_{n} \end{bmatrix}$ this doesn't hold because:

$\boldsymbol w^{T}\boldsymbol x = \begin{bmatrix} w_{1} \\ w_{2} \\ ... \\ w_{n} \end{bmatrix}\times \begin{bmatrix} x_{1} & x_{2} & ... & x_{n} \end{bmatrix} = \begin{bmatrix} w_{1}x_{1} & w_{1}x_{2} & ... & w_{1}x_{n} \\ w_{2}x_{1} & w_{2}x_{2} & ... & w_{2}x_{n} \\ ... & ... & ... & ... \\ w_{n}x_{1} & w_{n}x_{2} & ... &w_{n}x_{n} \end{bmatrix}$

Funny thing is that the EXACT reverse produces the desired output, that is a matrix with just one element being the above mentioned sum:

$\boldsymbol w\boldsymbol x^{T} = \begin{bmatrix} w_{1} & w_{2} & ... & w_{n} \end{bmatrix} \times \begin{bmatrix} x_{1} \\ x_{2} \\ ... \\ x_{n} \end{bmatrix} = \begin{bmatrix} w_{1}x_{1} + w_{2}x_{2} + ... + w_{n}x_{n} \end{bmatrix}$

So I guess the only explanation left is that both $\boldsymbol w$ and $\boldsymbol x$ are already in the "vertical" form:

$\boldsymbol w = \begin{bmatrix} w_{1} \\ w_{2} \\ ... \\ w_{n} \end{bmatrix} \ \ \ \boldsymbol x = \begin{bmatrix} x_{1} \\ x_{2} \\ ... \\ x_{n} \end{bmatrix}\ \ \$ which indeed makes $\ \ \boldsymbol w^{T} = \begin{bmatrix} w_{1} & w_{2} & ... & w_{n} \end{bmatrix}$

My question is: how do I know what vector / one-dimensional matrix direction the authors had in mind? Is is a convention that is ALWAYS followed that by default it's a "vertical" direction?

Or am I just supposed to guess based on knowing what the probable direction will be given the answer I expect? Here it's doable, but when I encounter more difficult equations it takes lots of time...

• yes, it is a general convention to represent vectors with a "table in vertical direction", i.e. with a $n \times 1$ matrix. – G Cab May 14 '16 at 18:20
• If $A$ is a matrix and $v$ is a vector, we usually want to write $Av$ for $A$ "applied" to $v$. This requires to have $v$ in column form. – Hagen von Eitzen May 14 '16 at 18:23
• Also, in your expansion of $WX^T$ you say it gives you the answer, but this is not the same answer that probably the book mentioned. You have evaluated to be a matrix multiplication , yielding a 1x1 matrix (or a row vector) -- it does not yield a scalar. It maybe trivial case of a 1x1 matrix/vector but it's not a scalar mathematically speaking. To obtain a scalar you specifically refer to dot product -- I am sure that is what the book is referring to. – Dhiraj Apr 3 '17 at 22:55

## 1 Answer

While column vectors are more common than row vectors, usually, books will mention whether they are using column vectors or row vectors. For example, they might write the following: $$\mathbf w=[1, 2, 3]^T$$ where the $T$ exponent means that this is the transpose of a row vector, making it a column vector. Although column vectors are usually more often referred to, linear algebra also makes use of row vectors sometimes, so make sure you are paying attention to the author's intentions and their mention of row/column vectors to make sure you know what kind of vector they are using.

• I see, thank you! This book doesn't really explain it anywhere, I looked for it but somehow the author has probably taken it for granted that it's the default of what to expect... – Jeremiah May 14 '16 at 18:27
• Hm...That's odd. Usually, at least in my experience, linear algebra books introduce what a column vector is and specify that they are using such, even if it's in a very small way like with the $T$. However, I've really only looked at two linear algebra books before, so I don't have that much experience. – Noble Mushtak May 14 '16 at 18:32
• It's not a linear algebra book, it's about statistics (regression). – Jeremiah May 14 '16 at 18:39