The following is the mathematical definition of a Artificial Neuron,

$$\textbf{Activation function } f: \qquad y = f(w^t x) = f\left( \sum_{i=1}^n w_i x_i \right)$$

Given that $W$ is a vector, what does the symbol $t$ represent?

What operation does it represent on a vector $w$?

  • $\begingroup$ $w^t x$ is the weighted average of $x.$ Were $w$ represents the weights. At least that is what it would be if all of the elements in $w$ are greater than $0$ and sum to $1.$ $\endgroup$ – Doug M Dec 21 '16 at 22:52
  • $\begingroup$ You used a lower-case $w$ and a capital $W$. If they both mean the same thing, you should use one or the other. $\endgroup$ – Michael Hardy Dec 21 '16 at 22:55

The $t$ stands for transpose. The result of the matrix multiplication between $w^t$ and $x$ is then simply the scalar/dot product between $w$ and $x$, which in turn is just $\sum_{i=1}^n w_ix_i$:

$$w^tx = \begin{pmatrix} w_1\\ w_2\\ \vdots\\ w_n\end{pmatrix}^t\begin{pmatrix} x_1\\ x_2\\ \vdots\\ x_n\end{pmatrix} = \begin{pmatrix} w_1 \; w_2 \; \ldots \;w_n \end{pmatrix}\begin{pmatrix} x_1\\ x_2\\ \vdots\\ x_n\end{pmatrix} = w \cdot n = \sum_{i=1}^n w_ix_i$$

  • $\begingroup$ Zillions of books say how matrix multiplication is defined, but nearly all of them omit to explain why it's defined that way, although that's easy to do. I wonder if some of that wouldn't help avert confusions about things like this. $\endgroup$ – Michael Hardy Dec 21 '16 at 22:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.