Combining Evolution and Learning in Computational Ecosystems


Although animals such as spiders, fish, and birds have very different anatomies, the basic mechanisms that govern their perception, decision-making, learning, reproduction, and death have striking similarities. These mechanisms have apparently allowed the development of general intelligence in nature. This led us to the idea of approaching artificial general intelligence (AGI) by constructing a generic artificial animal (animat) with a configurable body and fixed mechanisms of perception, decision-making, learning, reproduction, and death. One instance of this generic animat could be an artificial spider, another an artificial fish, and a third an artificial bird. The goal of all decision-making in this model is to maintain homeostasis. Thus actions are selected that might promote survival and reproduction to varying degrees. All decision-making is based on knowledge that is stored in network structures. Each animat has two such network structures: a genotype and a phenotype. The genotype models the initial nervous system that is encoded in the genome (“the brain at birth”), while the phenotype represents the nervous system in its present form (“the brain at present”). Initially the phenotype and the genotype coincide, but then the phenotype keeps developing as a result of learning, while the genotype essentially remains unchanged. The model is extended to ecosystems populated by animats that develop continuously according to fixed mechanisms for sexual or asexual reproduction, and death. Several examples of simple ecosystems are given. We show that our generic animat model possesses general intelligence in a primitive form. In fact, it can learn simple forms of locomotion, navigation, foraging, language, and arithmetic.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • Ackley, D., and Littman, M. 1992. Interactions between Learning and Evolution. In Langton, C. G.; Taylor, C.; Farmer, C. D.; and Rasmussen, S., eds., Artificial Life II, SFI Studies in the Sciences of Complexity. Reading, MA, USA: Addison-Wesley. 487–509.

  • Ashton, B. J.; Ridley, A. R.; Edwards, E. K.; and Thornton, A. 2018. Cognitive performance is linked to group size and affects fitness in Australian magpies. Nature 554(7692):364–367.

  • Avila-García, O., and Cañamero, L. 2005. Hormonal modulation of perception in motivation-based action selection architectures. In Proc. of the Symposium on Agents that Want and Like.

  • Bach, J. 2015. Modeling motivation in MicroPsi 2. In Proceedings of the 8th International Conference on Artificial General Intelligence, 3–13. Springer.

  • Boehm, A.-M., et al. 2012. FoxO is a critical regulator of stem cell maintenance in immortal Hydra. Proceedings of the National Academy of Sciences 109(48):19697–19702.

  • Bohannon, J. 2008. Flunking spore. Science 322(5901):531–531.

  • Bouneffouf, D.; Rish, I.; and Cecchi, G. 2017. Bandit Models of Human Behavior: Reward Processing in Mental Disorders. In Proc. of the 10th International Conference on Artificial General Intelligence, 237–248. Springer.

  • Braitenberg, V. 1986. Vehicles: Experiments in synthetic psychology. MIT press.

  • Buro, M. 1998. From simple features to sophisticated evaluation functions. In International Conference on Computers and Games, 126–145. Springer.

  • Carlsson, M. 2018. Animat Navigation Using Landmarks. Master’s thesis, Chalmers University of Technology.

  • Christensen, V., and Walters, C. J. 2004. Ecopath with Ecosim: methods, capabilities and limitations. Ecological modelling 172(2-4):109–139.

  • Coello, C. A. C.; Lamont, G. B.; Van Veldhuizen, D. A.; et al. 2007. Evolutionary algorithms for solving multi-objective problems, volume 5. Springer.

  • Cressey, D. 2015. Tropical paradise inspires virtual ecology lab. Nature 517(7534):255–256.

  • Draganski, B., and May, A. 2008. Training-induced structural changes in the adult human brain. Behavioural brain research 192(1):137–142.

  • Ernst, D.; Geurts, P.; and Wehenkel, L. 2005. Tree-based batch mode reinforcement learning. Journal of Machine Learning Research 6(Apr):503–556.

  • Fahlman, S. E., and Lebiere, C. 1990. The cascade-correlation learning architecture. In Advances in neural information processing systems, 524–532.

  • Futuyma, D. 2009. Evolution. Sinauer Associates.

  • Gabbay, D. M.; Hodkinson, I.; and Reynolds, M. 1994. Temporal logic (vol. 1): mathematical foundations and computational aspects. Oxford University Press, Inc.

  • Grund Pihlgren, G., and Lallo, N. 2018. Rule-Based Sequence Learning Extension for Animats. Master’s thesis, Chalmers University of Technology.

  • Healy, S. D., and Jones, C. M. 2002. Animal learning and memory: an integration of cognition and ecology1. Zoology 105(4):321–327.

  • Hubbell, S. P. 2001. The unified neutral theory of biodiversity and biogeography. Monographs in population biology. Princeton University Press.

  • Johannesson, L.; Nilsson, M.; and Strannegård, C. 2018. Basic Language Learning in Artificial Animals. In Biologically Inspired Cognitive Architectures Meeting, 155–161. Springer.

  • Karakotsios, K., and Bremer, M. 1993. SimLife: The official strategy guide. Prima Pub.

  • Keramati, M., and Gutkin, B. 2011. A reinforcement learning theory for homeostatic regulation. In Advances in neural information processing systems, 82–90.

  • Keramati, M., and Gutkin, B. 2014. Homeostatic reinforcement learning for integrating reward collection and physiological stability. Elife 3.

  • Knuth, D. E. 1997. The Art of Computer Programming, volume 2. Pearson Education.

  • Koza, J. R. 1989. Hierarchical Genetic Algorithms Operating on Populations of Computer Programs. In IJCAI, volume 89, 768–774.

  • Koza, J. R. 1994. Genetic programming as a means for programming computers by natural selection. Statistics and computing 4(2):87–112.

  • Langton, C. G. 1997. Artificial life: An overview. MIT Press.

  • LeCun, Y.; Bengio, Y.; and Hinton, G. 2015. Deep learning. Nature 521(7553):436.

  • Lotka, A. J. 1925. Elements of Physical Biology, by Alfred J. Lotka. Williams & Wilkins.

  • Mäkeläinen, F.; Torén, H.; and Strannegård, C. 2018a. Code for Homeostatic Agents with Dynamic State Representation.

  • Mäkeläinen, F.; Torén, H.; and Strannegård, C. 2018b. Efficient Concept Formation in Large State Spaces. In Proc. of the 11th International Conference on Artificial General Intelligence, Prague, Czech Republic, 140–150. Springer.

  • Niv, Y. 2009. Reinforcement learning in the brain. Journal of Mathematical Psychology 53(3):139–154.

  • Roijers, D. M.; Vamplew, P.; Whiteson, S.; Dazeley, R.; et al. 2013. A Survey of Multi-Objective Sequential Decision-Making. J. Artif. Intell. Res.(JAIR) 48:67–113.

  • Russell, S. J., and Zimdars, A. 2003. Q-decomposition for reinforcement learning agents. In Proceedings of the 20th International Conference on Machine Learning (ICML-03), 656–663.

  • Rusu, A. A.; Rabinowitz, N. C.; Desjardins, G.; Soyer, H.; Kirkpatrick, J.; Kavukcuoglu, K.; Pascanu, R.; and Hadsell, R. 2016. Progressive neural networks. arXiv preprint arXiv:1606.04671.

  • Sims, K. 1994. Evolving virtual creatures. In Proceedings of the 21st annual conference on Computer graphics and interactive techniques, 15–22. ACM.

  • Strannegård, C.; Nizamani, A. R.; Juel, J.; and Persson, U. 2016. Learning and Reasoning in Unknown Domains. Journal of Artificial General Intelligence 7(1):104–127.

  • Strannegård, C.; Svangård, N.; Lindström, D.; Bach, J.; and Steunebrink, B. 2017. The Animat Path to Artificial General Intelligence. In Proceedings of the Workshop on Architectures for Generality and Autonomy 2017.

  • Such, F. P.; Madhavan, V.; Conti, E.; Lehman, J.; Stanley, K. O.; and Clune, J. 2017. Deep Neuroevolution: Genetic Algorithms are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning. arXiv preprint arXiv:1712.06567.

  • Sutton, R. S., and Barto, A. G. 1998. Reinforcement learning: An introduction. MIT press.

  • Vladimirov, N.; Mu, Y.; Kawashima, T.; Bennett, D. V.; Yang, C.-T.; Looger, L. L.; Keller, P. J.; Freeman, J.; and Ahrens, M. B. 2014. Light-sheet functional imaging in fictively behaving zebrafish. Nature methods 11(9):883.

  • Von Neumann, J., et al. 1951. The general and logical theory of automata. Cerebral mechanisms in behavior 1(41):1–2.

  • Watkins, C. J. C. H. 1989. Learning from delayed rewards. Ph.D. Dissertation, King’s College, Cambridge.

  • Wilson, S. W. 1986. Knowledge growth in an artificial animal. In Adaptive and Learning Systems. Springer. 255–264.

  • Wilson, S. W. 1991. The animat path to AI. In Meyer, J. A., and Wilson, S. W., eds., From animals to animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior, 15–21. MIT Press.

  • Yaeger, L. 1994. Computational Genetics, Physiology, Metabolism, Neural Systems, Learning, Vision, and Behavior or PolyWorld: Life in a New Context. In Proceedings of the Artificial Life III conference, 263–298. Addison-Wesley.

  • Yoshida, N. 2017. Homeostatic Agent for General Environment. Journal of Artificial General Intelligence 8(1):1–22.


Journal + Issues