0
$\begingroup$

Hi I’ve recently started reading a book about deep learning (machine learning). It talks about multiplying vectors using dot product and elementwise multiplication. For instance, taking 3 inputs(vector) and 3 weights(vector) and multiplying. From my understanding, when multiplying vectors the size needs to be identical. The concept I’m have hard time understanding is multiplying vectors by a matrix. The book gives an example of an 1x4(vector) being multipled by 4x3(matrix). The output is an 1x3 vector. Im am confused because I assumed multiplying vector by matrix needs the same number of columns as well but have read that the matrixes need rows equal to the vectors columns. This is confusing to me because if I do not have equal number of columns how does my deep learning algorithm multiply each input in my vector by a corresponding weight?

$\endgroup$
0
$\begingroup$

For any matrix product $C=AB$, think of the element $c_{ij}$ as the dot product of the $i$th row of $A$ with the $j$th column of $B$. For that dot product to make sense, the “inner” dimensions of the matrix product—the number of columns of $A$ and the number of rows of $B$—have to match. The output matrix $C$ will have the “outer” dimensions—the same number of rows as $A$ and the same number of columns as $B$. You might think of these inner dimensions of $A$ and $B$ “canceling” when you multiply the matrices together.

$\endgroup$
  • $\begingroup$ ok understandable but in the deep learning algorithm examples; every input(vector) has a corresponding weight(vector). These vectors are are one row and multiple columns. How will the input get multiplied if vector matrix multiplication does not require that my columns are equal? $\endgroup$ – z Eyeland Feb 9 at 2:43
  • $\begingroup$ Isnt vector matrix multiplication just elementwise multiplication done to each row of the matrix? And elemntwise multiplication requires an equal amount of columns. $\endgroup$ – z Eyeland Feb 9 at 2:46
  • $\begingroup$ I the deep learning algorithm; I might have vector of three(columns) inputs. I also would have three weights. Each weight would have a vector of three columns to correspond With each input. These weights would be stored in matrix. The box gives an example of element wise multiplication with a 1x4 times 2x5 and states they don’t have same number of columns so it throws error. On the next page it give example of 1x4 times 4x3 and states the output is a 1x3 vector. This is contradicting. $\endgroup$ – z Eyeland Feb 9 at 3:19
  • $\begingroup$ @zEyeland Not at all. In fact, it illustrates my explanation: the first product is rejected because $4\ne5$; the second one makes sense because $4=4$, and the result has the “outer” dimensions, $1\times3$. Be aware, though, that there are other types of matrix products besides this standard one, such as the Hadamard product, which is the element-by-element product you seem to have in mind. $\endgroup$ – amd Feb 9 at 3:50
  • $\begingroup$ Am I suppose to flip the rows and columns of a matrix? $\endgroup$ – z Eyeland Feb 9 at 3:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.