5
$\begingroup$

As a recent field transferee from chemist to data scientist, I find myself wading through more matrix multiplication than I'm used to. I did some linear algebra way back, but I struggle with identifying 'which way' (i.e. inner or outer product) a given matrix multiplication is going.

I feel like there's some sort of convention with vector multiplication where the vector $\mathbf X$ is treated as a column vector, and $\mathbf X^T \mathbf X$ and $\mathbf X\mathbf X^T$ are the inner and outer product of $\mathbf X$ respectively. Is this true in all cases? Are there any tricks or mnemonics to help me keep track of the product of a string of such multiplications?

EDIT: There's a comment stating that it is common to assume that vectors are treated as column matrices. How common? Does it vary between disciplines? How likely (given that i'm looking through Wikipedia and stackexchange, not centuries-old manuscripts) am I to encounter the converse scenario?

$\endgroup$
3
  • 1
    $\begingroup$ Yes, it is common to assume that (coordinate) vectors $X$ are written as columns, or $n\times 1$ matrices ($n$ rows, $1$ column). Therefore, $X^T$ is then a $1\times n$ matrix. From the definition of matrix multiplication, one can multiply an $1\times n$ matrix by a $n\times 1$ one resulting in an $1\times 1$ matrix, which is identified with the single scalar that it contains. Multiplying a $n\times1$ matrix by a $1\times n$ matrix is also allowed, and it results in an $n\times n$ matrix. $\endgroup$
    – user580373
    Aug 1, 2018 at 23:48
  • 1
    $\begingroup$ @spiralstotheleft That really ought to be an answer instead of a comment. $\endgroup$
    – amd
    Aug 2, 2018 at 0:00
  • $\begingroup$ Yeah. Turn it into an answer and I will upvote it. $\endgroup$
    – Ingolifs
    Aug 2, 2018 at 0:30

1 Answer 1

Reset to default
8
$\begingroup$

From experience, column vectors aren't capitalized. Notes from wiki.

$X^{T}X $ and $XX^{T} $ denote the covariance matrix of $X \in \mathbb{C}^{m \times n}$

where as,

$ x^{t}x = \langle x, x \rangle = \| x\| $ is in the inner product and the outer product is given by $ x \otimes x = xx^{t} = \begin{bmatrix} x_{1} \\ x_{2} \\ x_{3} \end{bmatrix} \begin{bmatrix} x_{1} & x_{2} & x_{3} \end{bmatrix} = \begin{bmatrix} x_{1}x_{1} & x_{1}x_{2} & x_{1}x_{3} \\ x_{2}x_{1} & x_{2}x_{2} & x_{2}x_{3}\\ x_{3}x_{1} & x_{3}x_{2} & x_{3}x_{3}\end{bmatrix} $

Of course the outer product is for larger vectors as well i.e.

$$ u = (u_{1},u_{2}, \cdots,u_{m})\\ v= (v_{1},v_{2}, \cdots,v_{n}) $$

$$ u \otimes v = A = \begin{bmatrix} u_{1}v_{1} & u_{1}v_{2} & \cdots & u_{1}v_{n} \\ u_{2}v_{1} & u_{2}v_{2} & \cdots & u_{2}v_{n} \\ \vdots & \vdots & \ddots & \vdots \\ u_{m}v_{1} & u_{m}v_{2} & \cdots & u_{m}v_{n} \end{bmatrix} $$

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .