# Dot Product, matrix multiplication etc

Found this in a book: $$H=max\{0,X\cdot W^{(1)}+b^{(1)}\}$$ $$max\{0,\begin{bmatrix} 0& 0 \\0 &1\\ 1 &0\\ 1&1\end{bmatrix} \cdot \begin{bmatrix} -0.4& 0.1 & 0.9 \\0.8 & -0.2 & -0.7\end{bmatrix} +\begin{bmatrix} 0.6&-0.4& -0.7 \end{bmatrix}= \begin{bmatrix} 0.6 & -0.4 & -0.7 \\ 1.4&-0.6&-1.4\\0.2& -0.3&0.2\\1&-0.5& -0.5\end{bmatrix}\}$$

What kind of multiplication is this? It's definitely not matrix multiplication. It's not point wise ($\odot$) and I guess you can't call that scalar product. Bit confusing since such a multiplication has never been mentioned before in the book.

• There are several negative entries in the result. Why wasn't the max constraint triggered?
– greg
Jun 13 '20 at 14:41
• @greg not in the end result. I think you might have overlooked the curly bracke Jun 13 '20 at 16:37

## 1 Answer

It seems to be that the term $b^{(1)} = \begin{bmatrix} 0.6&-0.4& -0.7 \end{bmatrix}$ is simply shorthand for: $$\begin{bmatrix} 0.6&-0.4& -0.7 \\0.6&-0.4& -0.7 \\0.6&-0.4& -0.7 \\0.6&-0.4& -0.7 \end{bmatrix}$$

and then you are left with regular matrix multiplication:

$$\begin{bmatrix} 0& 0 \\0 &1\\ 1 &0\\ 1&1\end{bmatrix} \cdot \begin{bmatrix} -0.4& 0.1 & 0.9 \\0.8 & -0.2 & -0.7\end{bmatrix} +\begin{bmatrix} 0.6&-0.4& -0.7 \\0.6&-0.4& -0.7 \\0.6&-0.4& -0.7 \\0.6&-0.4& -0.7 \end{bmatrix}= \begin{bmatrix} 0.6 & -0.4 & -0.7 \\ 1.4&-0.6&-1.4\\0.2& -0.3&0.2\\1&-0.5& -0.5\end{bmatrix}$$

• well yes i know. that's called broadcasting. but thats not the part im interested in. i wonder what kind of multiplication that is. Aug 7 '18 at 11:26
• The initial part is standard matrix multiplication of a $4\times 2$ by a $2\times 3$ matrix, yielding a $4\times 3$ matrix. Aug 7 '18 at 11:29
• ou... well, i see the problem now^^ so silly. thank you guys Aug 7 '18 at 13:14