1
$\begingroup$

I'm currently in an Artificial intelligence class and have been given a challenging homework problem. My Linear algebra isn't quite up to scratch, and the problem is based around simplifying the multiplication of some vectors.

Normally we have a solution model of the form $ y = wx+b $ that can perform linear regression.

During a discussion of multilayer perceptrons and neural networks which take the output of a neuron (function) and input it into another function, we were given the following question:

Simplify the following: $${w_3[w_2(w_1 \overrightarrow{x} + b_1) + b_2] + b_3}$$

I believe the intent of this question is to show that this does not add any complexity to the model, and can be reduced to something similar to this:

$$ \overrightarrow{w}\overrightarrow{x} + \overrightarrow{b} $$

Is this assumption correct? If so, what initial steps would I take to simplify this. I'd like to do the work myself so tips or some identity in linear algebra would suffice.

$\endgroup$
6
  • 1
    $\begingroup$ This is indeed a misleading notation. $w_1$ needs to be a vector (assuming that $b_1$ is a scalar) and $w_1 \vec{x}$ should be a dot product, otherwise you cannot sum $w_1 \vec{x}$ with $b_1$, that is, you cannot sum a vector with a scalar (at least, I have never seen the definition of such operation). $\endgroup$
    – user168764
    Commented May 23, 2019 at 23:34
  • $\begingroup$ nbro If the notation was $\overrightarrow{b_1}$, would that be possible? There may be a possibility that either I or my professor omitted it accidentally. $\endgroup$
    – Howard P
    Commented May 24, 2019 at 11:44
  • 1
    $\begingroup$ You can distribute a scalar over vector addition and you can also multiply a scalar by a vector. If $w_1$ is a scalar, $\vec{x}$ and $\vec{b_1}$ are vectors, what can you do? What about $b_2, b_2, w_2$ and $w_3$? What are they? $\endgroup$
    – user168764
    Commented May 24, 2019 at 11:47
  • $\begingroup$ $w_1, w_2, w_3$ are all parameters, or weights, given the input $\overrightarrow{x}$. That is my interpretation at least. $b_1, b_2, b_3$ are also parameters. Algebraically simplifying may suffice(?). $\endgroup$
    – Howard P
    Commented May 24, 2019 at 11:55
  • 1
    $\begingroup$ Ok, but are they scalars or vectors? $\endgroup$
    – user168764
    Commented May 24, 2019 at 12:05

1 Answer 1

1
$\begingroup$

Think about the distributive property of matrix multiplication.

$\endgroup$
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .