From Nodes to Networks: Evolving Recurrent Neural Networks

Created by W.Langdon from gp-bibliography.bib Revision:1.4549

  author =       "Aditya Rawal and Risto Miikkulainen",
  title =        "From Nodes to Networks: Evolving Recurrent Neural
  howpublished = "arXiv",
  year =         "2018",
  month =        "12 " # mar,
  volume =       "abs/1803.04439",
  keywords =     "genetic algorithms, genetic programming, ANN, LSTM",
  URL =          "",
  URL =          "",
  size =         "8 pages",
  abstract =     "Gated recurrent networks such as those composed of
                 Long Short-Term Memory (LSTM) nodes have recently been
                 used to improve state of the art in many sequential
                 processing tasks such as speech recognition and machine
                 translation. However, the basic structure of the LSTM
                 node is essentially the same as when it was first
                 conceived 25 years ago. Recently, evolutionary and
                 reinforcement learning mechanisms have been employed to
                 create new variations of this structure. This paper
                 proposes a new method, evolution of a tree-based
                 encoding of the gated memory nodes, and shows that it
                 makes it possible to explore new variations more
                 effectively than other methods. The method discovers
                 nodes with multiple recurrent paths and multiple memory
                 cells, which lead to significant improvement in the
                 standard language modelling benchmark task. The paper
                 also shows how the search process can be speeded up by
                 training an LSTM network to estimate performance of
                 candidate structures, and by encouraging exploration of
                 novel solutions. Thus, evolutionary design of complex
                 neural network structures promises to improve
                 performance of deep learning architectures beyond human
                 ability to do so.",
  notes =        "p2 'Genetic Programming (GP) is used to evolve such
                 node architectures'

                 Entered for 2018 HUMIES",

Genetic Programming entries for Aditya Rawal Risto Miikkulainen