2
$\begingroup$

I was doing some research about the simplest neural network that can model each logic gate. By simplest I mean:

  • no bias if possible
  • fewest number of layers
  • fewest number of neurons in each layer
  • no activation function if possible

Starting from a 1-neuron network with no bias, I came up with the following chart:

enter image description here

I noticed that all the "negated" gates, namely NOT, NOR, NAND, and XNOR need a bias. Also note those gates are the negations of IDENTITY, OR, AND and XOR respectively.

1) Is this observation true?

2) If it is, why does "negation" require a bias?

Rigorous proofs and intuitive explanations are both welcome.

Edit: since there seems to be some confusion about what architecture/type of network I am referring too, I am adding a link showing diagrams of what I have found are the corresponding "simplest neural networks":

http://onehoursreflection.blogspot.com/2017/06/summary-of-simplest-neural-nets-for.html

$\endgroup$
  • $\begingroup$ Explain how only a bias is needed for a NOT gate, 0 + bias = 1 and 1 + bias = 0 does not make sense. You need to use a weight in there at least ( I don't see you mentioning weights anywere?) $\endgroup$ – Thomas W Jun 20 '17 at 10:44
  • $\begingroup$ Of course you need weights... I never said you were only using a bias... $\endgroup$ – nayriz Jun 20 '17 at 11:17
  • $\begingroup$ It is unclear. $minus(x)= -x, x \in \mathbb{R}$ is linear so a single linear neuron is sufficient. $not(x) = -x, x\in \{-1,1\}$ is also linear. $or(x,y), (x,y) \in \{-1,1\}$ is non-linear : you need a non-linear (sigmoid) neuron with a bias. The same for $and(x,y)$. $xor(x,y) = or(and(x,-y),and(-x,y))$ needs 3 non-linear neurons (2 layers) @ThomasW $\endgroup$ – reuns Jun 20 '17 at 11:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.