The top answer in this question claims that it is possible to model the XOR gate using a 2-2-1 neural network (2 inputs, 2 neurons in the hidden layer) and no activation function. (Or an identity linear activation function if you prefer).

Is this true?

  • 1
    $\begingroup$ The term is linear activation function👌 $\endgroup$ – Preston Roy Jun 21 '17 at 3:59
  • $\begingroup$ @Joshua_T , according to Dr Andrej Karpathy, you can say both: " Unlike all layers in a Neural Network, the output layer neurons most commonly do not have an activation function (or you can think of them as having a linear identity activation function). Source: cs231n.github.io/neural-networks-1 $\endgroup$ – nayriz Jun 21 '17 at 5:52
  • $\begingroup$ Concerning the question, I think it is unnecessary to consider using a linear activation function. The reason being that you want binary output, it is best to use a hard limit (unit step) transfer function. There is a solution to this problem using a 2-2-1 architecture and hard limit activation functions. You can refer to section 11-4 of "Neural Network Design" by Hagan, which can be found on the web $\endgroup$ – Preston Roy Jun 21 '17 at 12:33
  • $\begingroup$ @Joshua_T, it's probably unnecessary to use a neural network to model the XOR gate in the first place. It's just a theoretical exercise which I tried to code, and was wondering why I was getting the same result with a 1 layer and a 2 layer network. In another question, someone stated that the problem was solvable and their answer came on top, but I believe it is incorrect, so I just wanted others to help me check. $\endgroup$ – nayriz Jun 21 '17 at 15:51


A 2-2-1 linear neural network is a function

\begin{align} f: &\mathbb{R}^2 \rightarrow \mathbb{R}\\ f(x) &= B (A x + a) + b \qquad\text{with } x, a \in \mathbb{R}^{2}, A \in \mathbb{R}^{2 \times 2}, B \in \mathbb{R}^{1 \times 2}, b \in \mathbb{R} \end{align}

This is equivalent to

\begin{align} f(x) &= BAx + Ba + b \\ &= Cx + c \qquad\qquad \text{with } C \in \mathbb{R}^{1 \times 2}, c \in \mathbb{R} \end{align}

Obviously, $f$ is a linear function. Also quite obviously, the XOR-problem is not linearly seperable. Hence $f$ can't solve the XOR-problem.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.