I'm noticing something strange, i followed an example of coding a fully connected neural network in 3 layers. it uses backpropagation, and it works great. For example using Sweep optimization (ea sweeping trough starting train variables) this program can resolve the Irish flower data set with 99.16% accuracy (as total of train and validation data. I think thats an extreme score and its even so that i can bread multiple neural nets with such a score. So those networks are working great. (standard neural nets like 3:4:3)

But i wanted to put them trough other tests as wel. Then i thought lets try calculating RGB to HSL color space. Turns out the networks score 100% if try
RGB to H
or RGB to S
or RGB to l

But one network doing it all ea RGB to HSL ... it seams impossible. Is there something fundamentally difficult different that makes that problem that different. I would have thought that the nodes would weight balance and eventually seperate channels binding right output to right hidden nodes. But somehow it doesnt happen even if i triple the hidden nodes.

the code i use is based on this code in case your interested https://gist.github.com/atifaziz/9462430 (although it hasnt the sweep optimizer probaply reaching 85% accuracy).

I'm new to neural networks, and i like to know if there is some specific reason as of why a 3 layer neural net seams hopeless in resolving RGB to HSL Where it takes random RGB values to output HSL values; while having only one output for H or S or L result in 100% resolving. Is RGB to HSL somehow a different category problem. It doesnt seam to matter how many hidden nodes i use (but i only have 1 hidden layer, i'm not yet understanding the coding math behind deep neural networks).

So in short is there some mathematical reason relating to the backpropagition method. that a 3 layer network (input-hidden-output) can not be resolved ? As I am curious to know why the network cannt resolve it.

Sweep training that i use takes hundreds of starting variations, and each of them gets trained 5000 epochs which is more then enough for the irish flower set (thats resolved within 100 epochs and optimal in 1000 epochs).

(better explained i want to know if there is some fundamental reason why it doesnt get resolved)


closed as unclear what you're asking by Shailesh, Macavity, JonMark Perry, draks ..., Lord Shark the Unknown May 31 '17 at 5:26

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • $\begingroup$ its still on hold ? i updated my question i think someone in this field would understand what i say. (but if your not in neural networks math then probably not) $\endgroup$ – user3800527 May 31 '17 at 9:07
  • $\begingroup$ This question might be more appropriate on a different stackexchange site which is specific to machine learning or neural nets. I don't see a specific example of the math you're not understanding. If there's an equation or problem (a math problem) then please update your question accordingly. $\endgroup$ – Neev Parikh May 31 '17 at 10:00
  • $\begingroup$ i'm not aware of more specific sites, as i think it comes down to backpropagition solving. where each out put node changes all hidden layers in a fully connected network, And so i wonder if its logically speaking impossible to reach a solution in which each output node reserves only a few hidden nodes. Is it a theoretical limit of BPP. since is the BPP neural network is based on error correction statistics i wonder if some mathematician might know this answer. $\endgroup$ – user3800527 May 31 '17 at 11:55

Browse other questions tagged or ask your own question.