Artificial Motivation for Cognitive Software Agents

Ryan J. McCall 1 , Stan Franklin 2 , Usef Faghihi 3 , Javier Snaider 4  and Sean Kugele 5
  • 1 Computer Science Department and Institute for Intelligent Systems, 38152, Memphis
  • 2 Institute for Intelligent Systems, 38152, Memphis
  • 3 University of Indianapolis, 46227, Indianapolis
  • 4 Computer Science Department and Institute for Intelligent Systems, 38152, Memphis
  • 5 Computer Science Department and Institute for Intelligent Systems, 38152, Memphis

Abstract

Natural selection has imbued biological agents with motivations moving them to act for survival and reproduction, as well as to learn so as to support both. Artificial agents also require motivations to act in a goal-directed manner and to learn appropriately into various memories. Here we present a biologically inspired motivation system, based on feelings (including emotions) integrated within the LIDA cognitive architecture at a fundamental level. This motivational system, operating within LIDA’s cognitive cycle, provides a repertoire of motivational capacities operating over a range of time scales of increasing complexity. These include alarms, appraisal mechanisms, appetence and aversion, and deliberation and planning.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • Alvarado, N., Adams, S. S., & Burbeck, S. (2002). The role of emotion in an architecture of mind. IBM Research.

  • Baars, B. (1988). A Cognitive Theory of Consciousness. Cambridge: Cambridge University Press.

  • Baars, B., & Franklin, S. (2003). How conscious experience and working memory interact. Trends in Cognitive Science, 7, 166–172.

  • Bach, J. (2003). The micropsi agent architecture. Paper presented at the Proceedings of ICCM-5, international conference on cognitive modeling, Bamberg, Germany.

  • Bach, J. (2009). Principles of Synthetic Intelligence: Psi: An Architecture of Motivated Cognition. Oxford: Oxford University Press.

  • Bach, J. (2012). Modeling Motivation and the Emergence of Affect in a Cognitive Agent Theoretical Foundations of Artificial General Intelligence (pp. 241-262): Springer.

  • Barto, A. G. (2007). Temporal difference learning. Scholarpedia, 2(11), 1604.

  • Belavkin, R. V. (2001a). Modelling the inverted-U effect with ACT-R. In Erik M. Altmann, Wayne D. Gray, A. Cleeremans & Christian D. Schunn (Eds.), Proceedings of the 2001 Fourth International Conference on Cognitive Modeling (pp. 296). Hillsdale, NJ Lawrence Erlbaum Associates.

  • Belavkin, R. V. (2001b). The role of emotion in problem solving. Paper presented at the Proceedings of the AISB’01 Symposium on emotion, cognition and affective computing, Heslington, York, England.

  • Berridge, K. C., & Kringelbach, M. L. (2008). Affective neuroscience of pleasure: reward in humans and animals. Psychopharmacology, 199(3), 457-480. doi: 10.1007/s00213-008-1099-6

  • Berridge, K. C., & Robinson, T. E. (1998). What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Research Reviews, 28(3), 309-369.

  • Bindra, D. (1978). How adaptive behavior is produced: a perceptual-motivational alternative to response reinforcements. Behavioral and Brain Sciences, 1(01), 41-52.

  • Bogacz, R., Usher, M., Zhang, J., & McClelland, J. L. (2007). Extending a biologically inspired model of choice: multi-alternatives, nonlinearity and value-based multidimensional choice. Philos Trans R Soc Lond B Biol Sci.

  • Breazeal, C. (1998). A Motivational System for Regulating Human-Robot Interaction. Paper presented at the AAAI98, Madison, WI.

  • Camras, L. A. (2011). Differentiation, dynamical integration and functional emotional development. Emotion Review, 3(2), 138-146.

  • Cañamero, D. (1997). Modeling motivations and emotions as a basis for intelligent behavior. Paper presented at the Proceedings of the first international conference on Autonomous agents.

  • Canamero, Lola D. (2003). Designing Emotions for Activity Selection in Autonomous Agents. In R. Trappl, P. Petta & S. Payr (Eds.), Emotions in Humans and Artifacts (pp. 115-148). Cambridge, MA: MIT Press.

  • Cannon, W. B. (1927). The James-Lange theory of emotions: A critical examination and an alternative theory. The American Journal of Psychology, 39(1/4), 106-124.

  • Cannon, W. B. (1929). Organization For Physiological Homeostasis. Physiol Rev., 9, 399-431.

  • Conway, M. (2001). Sensory–perceptual episodic memory and its context: autobiographical memory. Philos. Trans. R. Soc. Lond B., 356, 1375–1384.

  • D’Mello, S., Ramamurthy, U., Negatu, A., & Franklin, S. (2006). A Procedural Learning Mechanism for Novel Skill Acquisition. In T. Kovacs & James A. R. Marshall (Eds.), Proceeding of Adaptation in Artificial and Biological Systems, AISB’06 (Vol. 1, pp. 184–185). Bristol, England: Society for the Study of Artificial Intelligence and the Simulation of Behaviour.

  • Damasio, A. (2003). Looking for Spinoza: Joy, Sorrow and the Feeling Brain. New York: Harcourt.

  • Damasio, A. (1999). The Feeling of What Happens. New York: Harcourt Brace.

  • Daw, N., Niv, Y., & Dayan, P. (2005). Actions, policies, values, and the basal ganglia. In E. Bezard (Ed.), Recent Breakthroughs in Basal Ganglia Research.

  • Daw, N. D., Niv, Y., & Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. [Research Support, Non-U.S. Gov’t]. Nature Neuroscience, 8(12), 1704-1711. doi: 10.1038/nn1560

  • Dehaene, S., Changeux, J.-P., Naccache, L., Sackur, J., & Sergent, C. (2006). Conscious, preconscious, and subliminal processing: a testable taxonomy. Trends in Cognitive Sciences, 10, 204–211.

  • Diener, E. (1999). Introduction to the special section on the structure of emotion. Journal of personality and Social Psychology, 76(5), 803.

  • Dijkstra, T. M. H., Schöner, G., & Gielen, C. C. A. M. (1994). Temporal stability of the action-perception cycle for postural control in a moving visual environment. Experimental Brain Research, 97(3), 477-486.

  • Dong, D., & Franklin, S. (2014). Sensory Motor System: Modeling the process of action execution. Paper presented at the Proceedings of the 36th Annual Conference of the Cognitive Science Society.

  • Dong, D., & Franklin, S. (2015). A New Action Execution Module for the Learning Intelligent Distribution Agent (LIDA): The Sensory Motor System. Cognitive Computation. doi: 10.1007/s12559-015-9322-3.

  • Dorner, D., & Hille, K. (1995). Artificial souls: motivated emotional robots. Paper presented at the IEEE International Conference on Systems, Man and Cybernetics, Vancouver, BC, Canada.

  • Drescher, Gary L. (1991). Made-Up Minds: A Constructivist Approach to Artificial Intelligence. Cambridge, MA: MIT Press.

  • Ekman, P., Sorenson, E. R., & Friesen, W. V. (1969). Pan-cultural elements in facial displays of emotion. Science, 164(3875), 86-88.

  • Faghihi, U., McCall, R., & Franklin, S. (2012). A Computational Model of Attentional Learning in a Cognitive Agent. Biologically Inspired Cognitive Architectures, 2, 25-36.

  • Faghihi, U., Estey, C., McCall, R., & Franklin, S. (2015). A Cognitive Model Fleshes Out Kahneman’s Fast and Slow Systems. Biologically Inspired Cognitive Architectures, 11, 38-52.

  • Faghihi, U., Nkambou, R., Poirier, P., & Fournier-Viger, P. (2009). Emotional Learning and a Combined Centralist-Peripheralist Based Architecture for a More Efficient Cognitive Agent. Paper presented at the 7th IEEE International Conference on Industrial Technology (ICIT 2009).

  • Fellous, J.-M. (2004). From human emotions to robot emotions. Architectures for Modeling Emotion: Cross-Disciplinary Foundations, American Association for Artificial Intelligence, 39-46.

  • Fishbach, A., Roy, S. A., Bastianen, C., Miller, L. E., & Houk, J. C. (2005). Kinematic properties of on-line error corrections in the monkey. Experimental Brain Research, 164(4), 442–457.

  • Franklin, S. (1995). Artificial Minds. Cambridge, Ma: MIT Press.

  • Franklin, S. (2000). Deliberation and Voluntary Action in ‘Conscious’ Software Agents. Neural Network World, 10, 505–521

  • Franklin, S. (2003). IDA: A Conscious Artifact? Journal of Consciousness Studies, 10, 47–66.

  • Franklin, S., & Baars, B. (2010). Two Varieties of Unconscious Processes. In E. Perry, D. Collerton, H. Ashton & F. LeBeau (Eds.), New Horizons in the Neuuroscience of Consciousness (pp. 91–102). Amsterdam: John Benjamin.

  • Franklin, S., Baars, B. J., Ramamurthy, U., & Ventura, M. (2005). The Role of Consciousness in Memory. Brains, Minds and Media, 1, 1–38.

  • Franklin, S., & Graesser, A. C. (1997). Is it an Agent, or just a Program?: A Taxonomy for Autonomous Agents Intelligent Agents III (pp. 21–35). Berlin: Springer Verlag.

  • Franklin, S., Kelemen, A., & McCauley, L. (1998). IDA: A Cognitive Agent Architecture IEEE Conf on Systems, Man and Cybernetics (pp. 2646–2651). Menlo Park, CA: IEEE Press.

  • Franklin, S., Madl, T., D’Mello, S., & Snaider, J. (2014). LIDA: A Systems-level Architecture for Cognition, Emotion, and Learning. IEEE Transactions on Autonomous Mental Development., PP(99), 1 doi: 10.1109/TAMD.2013.2277589

  • Franklin, S., Madl, T., Strain, S., Faghihi, U., Dong, D., Kugele, S., . . . Chen, S. (2016). A LIDA cognitive model tutorial. Biologically Inspired Cognitive Architectures, 105-130. doi: 10.1016/j.bica.2016.04.003

  • Franklin, S., & Ramamurthy, U. (2006). Motivations, Values and Emotions: Three sides of the same coin Proceedings of the Sixth International Workshop on Epigenetic Robotics (Vol. 128, pp. 41–48). Paris, France: Lund University Cognitive Studies.

  • Franklin, S., Strain, S., Snaider, J., McCall, R., & Faghihi, U. (2012). Global Workspace Theory, its LIDA model and the underlying neuroscience. Biologically Inspired Cognitive Architectures, 1, 32-43. doi: 10.1016/j.bica.2012.04.001

  • Franklin, S., Strain, S., McCall, R., & Baars, B. (2013). Conceptual Commitments of the LIDA Model of Cognition. Journal of Artificial General Intelligence, 4(2), 1-22. doi:10.2478/jagi-2013-0002

  • Freeman, W. J. (2002). The limbic action-perception cycle controlling goal-directed animal behavior. Neural Networks, 3, 2249-2254.

  • Fum, D., & Stocco, A. (2004). Memory, Emotion, and Rationality: An ACT-R interpretation for Gambling Task results. Paper presented at the ICCM.

  • Fuster, J. M. (2004). Upper processing stages of the perception–action cycle. Trends in Cognitive Sciences, 8(4), 143-145.

  • Gallagher, M., McMahan, R. W., & Schoenbaum, G. (1999). Orbitofrontal cortex and representation of incentive value in associative learning. The Journal of neuroscience, 19(15), 6610-6614.

  • Gmytrasiewicz, P. J., & Lisetti, C. L. (2002). Emotions and personality in agent design and modeling Game theory and decision theory in agent-based systems (pp. 81-95): Springer.

  • Hoffman, D. D., Singh, M., & Prakash, C. (2015). The interface theory of perception. Psychonomic bulletin & review, 22(6), 1480-1506.

  • Hollerman, J., & Schultz, W. (1998). Dopamine Neruons Report an Error in the Temproal Prediction of Reward during Learning. Nature Neuroscience, 1, 304-309.

  • Huys, Q. J., Eshel, N., O’Nions, E., Sheridan, L., Dayan, P., & Roiser, J. P. (2012). Bonsai trees in your head: how the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Comput Biol, 8(3), e1002410.

  • James, W. (1884). II.—What is an emotion? Mind(34), 188-205.

  • James, W. (1890). The Principles of Psychology. Cambridge, MA: Harvard University Press.

  • Johnston, Victor S. (1999). Why We Feel:The Science of Human Emotions. Reading MA: Perseus Books.

  • Kahneman, D. (2003). Maps of bounded rationality: Psychology for behavioral economics. The American economic review, 93(5), 1449-1475.

  • Kahneman, D. (2011). Thinking, Fast and Slow. New York: Farrar, Straus and Giroux.

  • Kalis, A., Kaiser, S., & Mojzisch, A. (2013). Why we should talk about option generation in decision-making research. Front. Psychol, 4(555), 10.3389.

  • Keller, L. R., & Ho, J. L. (1988). Decision problem structuring: Generating options. Systems, Man and Cybernetics, IEEE Transactions on, 18(5), 715-728.

  • Klein, G., Wolf, S., Militello, L., & Zsambok, C. (1995). Characteristics of skilled option generation in chess. Organizational Behavior and Human Decision Processes, 62(1), 63-69.

  • Kringelbach, M. L., & Berridge, K. C. (2009). Towards a functional neuroanatomy of pleasure and happiness. Trends in cognitive sciences, 13(11), 479-487.

  • Laird, John E., J, E., Newell, A., & Rosenbloom, Paul S. P. S. (1987). SOAR: An Architecture for General Intelligence. Artificial Intelligence, 33, 1–64.

  • Lang, P. J., & Davis, M. (2006). Emotion, motivation, and the brain: Reflex foundations in animal and human research. In G. E. M. J. J. K. S. Anders & D. Wildgruber (Eds.), Progress in Brain Research (Vol. Volume 156, pp. 3-29): Elsevier.

  • Lazarus, R. (1991). Emotion and adaptation. New York: Oxford University Press.

  • LeDoux, J. E. (2006). Emotional Memory: In Search of Systems and Synapsesa. Annals of the New York Academy of Sciences, 702(1), 149-157.

  • Lee-Johnson, C. P., & Carnegie, D. A. (2009). Robotic Emotions: Navigation with Feeling. In J. Vallverdú & D. Casacuberta (Eds.), Handbook of Research on Synthetic Emotions and Sociable Robotics (pp. 88-117): IGI Global.

  • Liddell, B. J., Brown, K. J., Kemp, A. H., Barton, M. J., Das, P., Peduto, A., . . . Williams, L. M. (2005). A direct brainstem–ìamygdala–cortical ‘alarm’ system for subliminal signals of fear. NeuroImage, 24(1), 235-243.

  • Lucantonio, F., Stalnaker, T. A., Shaham, Y., Niv, Y., & Schoenbaum, G. (2012). The impact of orbitofrontal dysfunction on cocaine addiction. Nature Neuroscience, 15(3), 358-366.

  • MacDonald, K. (2008). Effortful Control, Explicit Processing and the Regulation of Human Evolved Predispositions. Psychological Review, 115(4), 012–1031.

  • Madl, T., Baars, B. J., & Franklin, S. (2011). The Timing of the Cognitive Cycle. PLoS ONE, 6(4), e14803.

  • Madl, T., & Franklin, S. (2012). A LIDA-based Model of the Attentional Blink. Proceedings of the 11th International Conference on Cognitive Modelling, 283-288.

  • Madl, T., Franklin, S., Chen, K., & Trappl, R. (2013). Spatial Working Memory in the LIDA Cognitive Architecture. In R. West & T. Stewart (Eds.), Proceedings of the 12th International Conference on Cognitive Modelling (pp. 384-390). Ottawa, Canada: Carleton University.

  • Maes, P. (1989). How to do the right thing. Connection Science, 1, 291–323.

  • Marieb, E. N., & Hoehn, K. (2007). Human Anatomy & Physiology (Seventh ed.). San Francisco, CA: Pearson Benjamin Cummings.

  • Marinier, R., & Laird, J. E. (2008). Emotion-driven reinforcement learning. Cognitive science, 115-120.

  • Marinier, R. P., Laird, J. E., & Lewis, R. L. (2009). A computational unification of cognitive behavior and emotion. Cognitive Systems Research, 10(1), 48-69.

  • McCall, R., Franklin, S., & Friedlander, D. (2010). Grounded Event-Based and Modal Representations for Objects, Relations, Beliefs, Etc. Paper presented at the FLAIRS-23, Daytona Beach, FL.

  • McCall, R. J. (2014). Fundamental motivation and perception for a systems-level cognitive architecture. The University of Memphis.

  • Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., ... & Petersen, S. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529.

  • Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. The Journal of neuroscience, 16(5), 1936-1947.

  • Negatu, A. (2006). Cognitively Inspired Decision Making for Software Agents: Integrated Mechanisms for Action Selection, Expectation, Automatization and Non-Routine Problem Solving: Ph.D. Dissertation, The University of Memphis, Memphis TN USA.

  • Neisser, U. (1976). Cognition and Reality: Principles and Implications of Cognitive Psychology San Francisco: W. H. Freeman.

  • O’Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron, 38(2), 329-337.

  • Pasquereau, B., Nadjar, A., Arkadir, D., Bezard, E., Goillandeau, M., Bioulac, B., . . . Boraud, T. (2007). Shaping of motor responses by incentive values through the basal ganglia. Journal of Neuroscience, 27, 1176-1183.

  • Phelps, E. A. (2006). Emotion and Cognition: Insights from Studies of the Human Amygdala. Annual Review of Psychology, 57(1), 27-53. doi: doi:10.1146/annurev.psych.56.091103.070234

  • Picard, R. (1997). Affective Computing. Cambridge MA: The MIT Press.

  • Picard, R. W. (2003). Affective computing: challenges. International Journal of Human-Computer Studies, 59(1‚Äì2), 55-64. doi: 10.1016/s1071-5819(03)00052-1

  • Purves, D., Brannon, E. M., Cabeza, R., Huettel, S. A., LaBar, K. S., Platt, M. L., & Woldorff, M. G. (2008). Principles of cognitive neuroscience (Vol. 83): Sinauer Associates Sunderland, MA.

  • Raab, M., de Oliveira, R. F., & Heinen, T. (2009). How do people perceive and generate options? Progress in brain research, 174, 49-59.

  • Richard, J. M., & Berridge, K. C. (2011). Nucleus accumbens dopamine/glutamate interaction switches modes to generate desire versus dread: D1 alone for appetitive eating but D1 and D2 together for fear. The Journal of neuroscience, 31(36), 12866-12879.

  • Roseman, I. J., & Smith, C. A. (2001). Appraisal theory: Overview, assumptions, varieties, controversies Appraisal processes in emotion: Theory, methods, research (pp. 3-19). New York: Oxford University Press.

  • Rowe, J., Hughes, L., Eckstein, D., & Owen, A. M. (2008). Rule-Selection and Action-Selection have a Shared Neuroanatomical Basis in the Human Prefrontal and Parietal Cortex. Cerebral Cortex, 18, 2275-2285. doi: 10.1093/cercor/bhm249

  • Schoenbaum, G., Takahashi, Y., Liu, T. L., & McDannald, M. A. (2011). Does the orbitofrontal cortex signal value? Annals of the New York Academy of Sciences, 1239(1), 87-99.

  • Shin, Y. K., Proctor, R. W., & Capaldi, E. J. (2010). A review of contemporary ideomotor theory. Psychological Bulletin, 136(6), 943-974. doi: 10.1037/a0020541

  • Sloman, A. (1998). Damasio, Descartes, Alarms and Meta-management Proceedings Symposiumon Cognitive Agents: Modeling Human Cognition. San Diego: IEEE.

  • Sloman, A. (1999). What Sort of Architecture is Required for a Human-like Agent? In M. Wooldridge & A. S. Rao (Eds.), Foundations of Rational Agency (pp. 35–52). Dordrecht, Netherlands: Kluwer Academic Publishers.

  • Sloman, A., & Croucher, M. (1981). Why robots will have emotions.

  • Smith, K. S., Berridge, K. C., & Aldridge, J. W. (2011). Disentangling pleasure from incentive salience and learning signals in brain reward circuitry. Proceedings of the National Academy of Sciences, 108(27), E255-E264.

  • Snaider, J., McCall, R., & Franklin, S. (2011). The LIDA Framework as a General Tool for AGI. Paper presented at the Artificial General Intelligence (AGI-11), Mountain View, CA.

  • Squire, L. R., & Kandel, E. R. (2000). Memory: From mind to molecules: Macmillan.

  • Sun, R. (2009). Motivational representations within a computational cognitive architecture. Cognitive Computation, 1(1), 91-103.

  • Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.

  • Thompson, R. F., & Madigan, S. A. (2007). Memory. Princeton, NJ: Princeton University Press.

  • Tindell, A. J., Smith, K. S., Berridge, K. C., & Aldridge, J. W. (2009). Dynamic computation of incentive salience: ‘wanting’ what was never ‘liked’. The Journal of Neuroscience, 29(39), 12220-12228.

  • Toates, F. M. (1986). Motivational systems: CUP Archive.

  • Ward, P., Suss, J., Eccles, D. W., Williams, A. M., & Harris, K. R. (2011). Skill-based differences in option generation in a complex task: A verbal protocol analysis. Cognitive processing, 12(3), 289-300.

  • Watkins, C. J. C. H. (1989). Learning from Delayed Rewards. Ph.D. thesis, Cambridge University.

  • Westen, D. (1999). The Scientific Status of Unconscious Processes: Is Freud Really Dead? Journal of the American Psychoanalytic Association, 47(4), 1061-1106. doi: 10.1177/000306519904700404

  • Wimmer, G. E., & Shohamy, D. (2012). Preference by Association: How Memory Mechanisms in the Hippocampus Bias Decisions. Science, 338(6104), 270-273. doi: 10.1126/science.1223252

  • Yerkes, R. M., & Dodson, J. D. (1908). The Relationship of Strength of Stimulus to Rapidity of Habit Formation. Journal of Comparative Neurology and Psychology, 18, 459–482.

  • Zacks, J. M., Speer, N. K., Swallow, K. M., Braver, T. S., & Reynolds, J. R. (2007). Event Perception: A Mind–Brain Perspective. Psychological Bulletin, 133(2), 273–293.

  • Zacks, J. M., & Tversky, B. (2001). Event structure in perception and conception. Psychological bulletin, 127(1), 3.

OPEN ACCESS

Journal + Issues

Search