Evolving Neural Network using both genetic algorithm and back propagation I didn't find any literature on this matter; I want to build an RNN that uses both Genetic algorithms and back propagation to evolve a network. So my idea is to use GA and BP for training, though in a newer way:


*

*For optimization of the weights we will use BP. 

*For creating new layers, new edges and nodes in the graph using mutation and fitness, namely GA.
What do you think about this approach, why people haven not done it before? 
 A: The main issue I see with your proposal is that each time the genetic algorithm changes the network architecture, it will render all the backprop learning done previously somewhat useless (maybe not entirely, but backprop is always computed wrt the current network structure). 
Also,  training deep NNs is quite expensive and generally stochastic (meaning you should ideally run multiple trainings for each network architecture change... costly!). 
It's not obvious that GAs are the most efficient way to do this.
Bayesian optimization seems likely to be more efficient IMO.
It just seems far easier to make an overly large, well-regularized network and backprop, and let the learning process figure it out.
That being said, people have been trying things like what you are saying. Here are some related works:


*

*Miller et al, Designing Neural Networks using Genetic Algorithms., 1989

*Young et al, Optimizing Deep Learning Hyper-Parameters Through an Evolutionary Algorithm, 2015

*Xie and Yuille, Genetic CNN, 2017

*Fernando et al, PathNet: Evolution Channels Gradient Descent in Super Neural Networks, 2017

*Real et al, Large-Scale Evolution of Image Classifiers, 2017
A relatively famous paper using evolutionary approaches to train the network (not doing structure search, but rather training in reinforcement learning) is Such et al, Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning, 2018.
In general, the area you are talking about is called architecture search. Some papers that might interest you:


*

*Liu et al, Progressive neural architecture search, 2017

*Liu et al, Hierarchical representations for efficient architecture search, 2017

*Brock et al, SMASH: one-shot model architecture search through hypernetworks, 2017
Most of these techniques apply to RNNs quite straightforwardly. 
