0
$\begingroup$

The top answer in this question claims that it is possible to model the XOR gate using a 2-2-1 neural network (2 inputs, 2 neurons in the hidden layer) and no activation function. (Or an identity linear activation function if you prefer).

Is this true?

$\endgroup$
4
  • 1
    $\begingroup$ The term is linear activation function👌 $\endgroup$ Jun 21, 2017 at 3:59
  • $\begingroup$ @Joshua_T , according to Dr Andrej Karpathy, you can say both: " Unlike all layers in a Neural Network, the output layer neurons most commonly do not have an activation function (or you can think of them as having a linear identity activation function). Source: cs231n.github.io/neural-networks-1 $\endgroup$
    – nayriz
    Jun 21, 2017 at 5:52
  • $\begingroup$ Concerning the question, I think it is unnecessary to consider using a linear activation function. The reason being that you want binary output, it is best to use a hard limit (unit step) transfer function. There is a solution to this problem using a 2-2-1 architecture and hard limit activation functions. You can refer to section 11-4 of "Neural Network Design" by Hagan, which can be found on the web $\endgroup$ Jun 21, 2017 at 12:33
  • $\begingroup$ @Joshua_T, it's probably unnecessary to use a neural network to model the XOR gate in the first place. It's just a theoretical exercise which I tried to code, and was wondering why I was getting the same result with a 1 layer and a 2 layer network. In another question, someone stated that the problem was solvable and their answer came on top, but I believe it is incorrect, so I just wanted others to help me check. $\endgroup$
    – nayriz
    Jun 21, 2017 at 15:51

1 Answer 1

1
$\begingroup$

No.

A 2-2-1 linear neural network is a function

\begin{align} f: &\mathbb{R}^2 \rightarrow \mathbb{R}\\ f(x) &= B (A x + a) + b \qquad\text{with } x, a \in \mathbb{R}^{2}, A \in \mathbb{R}^{2 \times 2}, B \in \mathbb{R}^{1 \times 2}, b \in \mathbb{R} \end{align}

This is equivalent to

\begin{align} f(x) &= BAx + Ba + b \\ &= Cx + c \qquad\qquad \text{with } C \in \mathbb{R}^{1 \times 2}, c \in \mathbb{R} \end{align}

Obviously, $f$ is a linear function. Also quite obviously, the XOR-problem is not linearly seperable. Hence $f$ can't solve the XOR-problem.

$\endgroup$
1
  • $\begingroup$ From a modelling perspective, it is generally accepted that nonlinearity refers to when "observational data are modelled by a function which is a nonlinear combination of the model parameters and depends on one or more independent variables."[1] With that in mind, can you elaborate your point given the above in view of the $BAx$ term in the f function. [1] en.wikipedia.org/wiki/Nonlinear_regression $\endgroup$
    – hH1sG0n3
    Jul 11, 2022 at 13:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .