FANDOM


The goal, as far as I understand, is to escape from the current simple execution of neural networks and have them work out programs by themselves. The ultimate product of this line of research will be a Gödel machine. It is actively carried out by Google and Facebook employees, among others.

TODO: Important development: neural stored-program memory (Le et al. 2020)[1] approximates universal Turing machine.

  1. Neural Turing Machine (NTM) (Graves et al., 2014)[2]
  2. Stack-Augmented recurrent neural networks (Joulin & Mikolov, 2015[3]; Grefenstette et al., 2015[4])
  3. Grid-LSTM (Kalchbrenner et al., 2015)[5]
  4. Neural Random-access machines: Kurach et al. (2015)[6]
  5. Zaremba et al. (2015)[7]
  6. Neural programmer: Neelakantan et al. (2015)[8]
  7. Neural GPUs: Kaiser & Sutskever (2015)[9]
  8. Zhang et al. (2015)[10]
  9. Neural Programmer-Interpreters: Reed & de Freitas (2015)[11]

References Edit

  1. Le, H., Tran, T., & Venkatesh, S. (2020). Neural Stored-program Memory. In ICLR 2020.
  2. Graves, Alex, Wayne, Greg, and Danihelka, Ivo. Neural turing machines. arXiv preprint arXiv:1410.5401, 2014.
  3. Joulin, Armand and Mikolov, Tomas. Inferring algorithmic patterns with stack-augmented recurrent nets. arXiv preprint arXiv:1503.01007, 2015.
  4. Joulin, Armand and Mikolov, Tomas. Inferring algorithmic patterns with stack-augmented recurrent nets. arXiv preprint arXiv:1503.01007, 2015.
  5. Kalchbrenner, Nal, Danihelka, Ivo, and Graves, Alex. Grid long short-term memory. arXiv preprint arXiv:1507.01526, 2015
  6. Karol Kurach, Marcin Andrychowicz, Ilya Sutskever. 2015. NEURAL RANDOM-ACCESS MACHINES Arxiv preprint
  7. LEARNING SIMPLE ALGORITHMS FROM EXAMPLES. 2015. Wojciech Zaremba, Tomas Mikolov, Armand Joulin, Rob Fergus
  8. NEURAL PROGRAMMER: INDUCING LATENT PROGRAMS WITH GRADIENT DESCENT. Arvind Neelakantan, Quoc V. Le, Ilya Sutskever
  9. Łukasz Kaiser & Ilya Sutskever. 2015. NEURAL GPUS LEARN ALGORITHMS
  10. Wei Zhang, Yang Yu, Bowen Zhou. 2015. Structured Memory for Neural Turing Machines Arxiv
  11. Scott Reed and Nando de Freitas. 2015. Neural Programmer-Interpreters
Community content is available under CC-BY-SA unless otherwise noted.