Artificial Motivation for Cognitive Software Agents

Open access

Abstract

Natural selection has imbued biological agents with motivations moving them to act for survival and reproduction, as well as to learn so as to support both. Artificial agents also require motivations to act in a goal-directed manner and to learn appropriately into various memories. Here we present a biologically inspired motivation system, based on feelings (including emotions) integrated within the LIDA cognitive architecture at a fundamental level. This motivational system, operating within LIDA’s cognitive cycle, provides a repertoire of motivational capacities operating over a range of time scales of increasing complexity. These include alarms, appraisal mechanisms, appetence and aversion, and deliberation and planning.

If the inline PDF is not rendering correctly, you can download the PDF file here.

  • Alvarado N. Adams S. S. & Burbeck S. (2002). The role of emotion in an architecture of mind. IBM Research.

  • Baars B. (1988). A Cognitive Theory of Consciousness. Cambridge: Cambridge University Press.

  • Baars B. & Franklin S. (2003). How conscious experience and working memory interact. Trends in Cognitive Science 7 166–172.

  • Bach J. (2003). The micropsi agent architecture. Paper presented at the Proceedings of ICCM-5 international conference on cognitive modeling Bamberg Germany.

  • Bach J. (2009). Principles of Synthetic Intelligence: Psi: An Architecture of Motivated Cognition. Oxford: Oxford University Press.

  • Bach J. (2012). Modeling Motivation and the Emergence of Affect in a Cognitive Agent Theoretical Foundations of Artificial General Intelligence (pp. 241-262): Springer.

  • Barto A. G. (2007). Temporal difference learning. Scholarpedia 2(11) 1604.

  • Belavkin R. V. (2001a). Modelling the inverted-U effect with ACT-R. In Erik M. Altmann Wayne D. Gray A. Cleeremans & Christian D. Schunn (Eds.) Proceedings of the 2001 Fourth International Conference on Cognitive Modeling (pp. 296). Hillsdale NJ Lawrence Erlbaum Associates.

  • Belavkin R. V. (2001b). The role of emotion in problem solving. Paper presented at the Proceedings of the AISB’01 Symposium on emotion cognition and affective computing Heslington York England.

  • Berridge K. C. & Kringelbach M. L. (2008). Affective neuroscience of pleasure: reward in humans and animals. Psychopharmacology 199(3) 457-480. doi: 10.1007/s00213-008-1099-6

  • Berridge K. C. & Robinson T. E. (1998). What is the role of dopamine in reward: hedonic impact reward learning or incentive salience? Brain Research Reviews 28(3) 309-369.

  • Bindra D. (1978). How adaptive behavior is produced: a perceptual-motivational alternative to response reinforcements. Behavioral and Brain Sciences 1(01) 41-52.

  • Bogacz R. Usher M. Zhang J. & McClelland J. L. (2007). Extending a biologically inspired model of choice: multi-alternatives nonlinearity and value-based multidimensional choice. Philos Trans R Soc Lond B Biol Sci.

  • Breazeal C. (1998). A Motivational System for Regulating Human-Robot Interaction. Paper presented at the AAAI98 Madison WI.

  • Camras L. A. (2011). Differentiation dynamical integration and functional emotional development. Emotion Review 3(2) 138-146.

  • Cañamero D. (1997). Modeling motivations and emotions as a basis for intelligent behavior. Paper presented at the Proceedings of the first international conference on Autonomous agents.

  • Canamero Lola D. (2003). Designing Emotions for Activity Selection in Autonomous Agents. In R. Trappl P. Petta & S. Payr (Eds.) Emotions in Humans and Artifacts (pp. 115-148). Cambridge MA: MIT Press.

  • Cannon W. B. (1927). The James-Lange theory of emotions: A critical examination and an alternative theory. The American Journal of Psychology 39(1/4) 106-124.

  • Cannon W. B. (1929). Organization For Physiological Homeostasis. Physiol Rev. 9 399-431.

  • Conway M. (2001). Sensory–perceptual episodic memory and its context: autobiographical memory. Philos. Trans. R. Soc. Lond B. 356 1375–1384.

  • D’Mello S. Ramamurthy U. Negatu A. & Franklin S. (2006). A Procedural Learning Mechanism for Novel Skill Acquisition. In T. Kovacs & James A. R. Marshall (Eds.) Proceeding of Adaptation in Artificial and Biological Systems AISB’06 (Vol. 1 pp. 184–185). Bristol England: Society for the Study of Artificial Intelligence and the Simulation of Behaviour.

  • Damasio A. (2003). Looking for Spinoza: Joy Sorrow and the Feeling Brain. New York: Harcourt.

  • Damasio A. (1999). The Feeling of What Happens. New York: Harcourt Brace.

  • Daw N. Niv Y. & Dayan P. (2005). Actions policies values and the basal ganglia. In E. Bezard (Ed.) Recent Breakthroughs in Basal Ganglia Research.

  • Daw N. D. Niv Y. & Dayan P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. [Research Support Non-U.S. Gov’t]. Nature Neuroscience 8(12) 1704-1711. doi: 10.1038/nn1560

  • Dehaene S. Changeux J.-P. Naccache L. Sackur J. & Sergent C. (2006). Conscious preconscious and subliminal processing: a testable taxonomy. Trends in Cognitive Sciences 10 204–211.

  • Diener E. (1999). Introduction to the special section on the structure of emotion. Journal of personality and Social Psychology 76(5) 803.

  • Dijkstra T. M. H. Schöner G. & Gielen C. C. A. M. (1994). Temporal stability of the action-perception cycle for postural control in a moving visual environment. Experimental Brain Research 97(3) 477-486.

  • Dong D. & Franklin S. (2014). Sensory Motor System: Modeling the process of action execution. Paper presented at the Proceedings of the 36th Annual Conference of the Cognitive Science Society.

  • Dong D. & Franklin S. (2015). A New Action Execution Module for the Learning Intelligent Distribution Agent (LIDA): The Sensory Motor System. Cognitive Computation. doi: 10.1007/s12559-015-9322-3.

  • Dorner D. & Hille K. (1995). Artificial souls: motivated emotional robots. Paper presented at the IEEE International Conference on Systems Man and Cybernetics Vancouver BC Canada.

  • Drescher Gary L. (1991). Made-Up Minds: A Constructivist Approach to Artificial Intelligence. Cambridge MA: MIT Press.

  • Ekman P. Sorenson E. R. & Friesen W. V. (1969). Pan-cultural elements in facial displays of emotion. Science 164(3875) 86-88.

  • Faghihi U. McCall R. & Franklin S. (2012). A Computational Model of Attentional Learning in a Cognitive Agent. Biologically Inspired Cognitive Architectures2 25-36.

  • Faghihi U. Estey C. McCall R. & Franklin S. (2015). A Cognitive Model Fleshes Out Kahneman’s Fast and Slow Systems. Biologically Inspired Cognitive Architectures 11 38-52.

  • Faghihi U. Nkambou R. Poirier P. & Fournier-Viger P. (2009). Emotional Learning and a Combined Centralist-Peripheralist Based Architecture for a More Efficient Cognitive Agent. Paper presented at the 7th IEEE International Conference on Industrial Technology (ICIT 2009).

  • Fellous J.-M. (2004). From human emotions to robot emotions. Architectures for Modeling Emotion: Cross-Disciplinary Foundations American Association for Artificial Intelligence 39-46.

  • Fishbach A. Roy S. A. Bastianen C. Miller L. E. & Houk J. C. (2005). Kinematic properties of on-line error corrections in the monkey. Experimental Brain Research 164(4) 442–457.

  • Franklin S. (1995). Artificial Minds. Cambridge Ma: MIT Press.

  • Franklin S. (2000). Deliberation and Voluntary Action in ‘Conscious’ Software Agents. Neural Network World 10 505–521

  • Franklin S. (2003). IDA: A Conscious Artifact? Journal of Consciousness Studies 10 47–66.

  • Franklin S. & Baars B. (2010). Two Varieties of Unconscious Processes. In E. Perry D. Collerton H. Ashton & F. LeBeau (Eds.) New Horizons in the Neuuroscience of Consciousness (pp. 91–102). Amsterdam: John Benjamin.

  • Franklin S. Baars B. J. Ramamurthy U. & Ventura M. (2005). The Role of Consciousness in Memory. Brains Minds and Media 1 1–38.

  • Franklin S. & Graesser A. C. (1997). Is it an Agent or just a Program?: A Taxonomy for Autonomous Agents Intelligent Agents III (pp. 21–35). Berlin: Springer Verlag.

  • Franklin S. Kelemen A. & McCauley L. (1998). IDA: A Cognitive Agent Architecture IEEE Conf on Systems Man and Cybernetics (pp. 2646–2651). Menlo Park CA: IEEE Press.

  • Franklin S. Madl T. D’Mello S. & Snaider J. (2014). LIDA: A Systems-level Architecture for Cognition Emotion and Learning. IEEE Transactions on Autonomous Mental Development. PP(99) 1 doi: 10.1109/TAMD.2013.2277589

  • Franklin S. Madl T. Strain S. Faghihi U. Dong D. Kugele S. . . . Chen S. (2016). A LIDA cognitive model tutorial. Biologically Inspired Cognitive Architectures 105-130. doi: 10.1016/j.bica.2016.04.003

  • Franklin S. & Ramamurthy U. (2006). Motivations Values and Emotions: Three sides of the same coin Proceedings of the Sixth International Workshop on Epigenetic Robotics (Vol. 128 pp. 41–48). Paris France: Lund University Cognitive Studies.

  • Franklin S. Strain S. Snaider J. McCall R. & Faghihi U. (2012). Global Workspace Theory its LIDA model and the underlying neuroscience. Biologically Inspired Cognitive Architectures 1 32-43. doi: 10.1016/j.bica.2012.04.001

  • Franklin S. Strain S. McCall R. & Baars B. (2013). Conceptual Commitments of the LIDA Model of Cognition. Journal of Artificial General Intelligence 4(2) 1-22. doi:10.2478/jagi-2013-0002

  • Freeman W. J. (2002). The limbic action-perception cycle controlling goal-directed animal behavior. Neural Networks 3 2249-2254.

  • Fum D. & Stocco A. (2004). Memory Emotion and Rationality: An ACT-R interpretation for Gambling Task results. Paper presented at the ICCM.

  • Fuster J. M. (2004). Upper processing stages of the perception–action cycle. Trends in Cognitive Sciences 8(4) 143-145.

  • Gallagher M. McMahan R. W. & Schoenbaum G. (1999). Orbitofrontal cortex and representation of incentive value in associative learning. The Journal of neuroscience 19(15) 6610-6614.

  • Gmytrasiewicz P. J. & Lisetti C. L. (2002). Emotions and personality in agent design and modeling Game theory and decision theory in agent-based systems (pp. 81-95): Springer.

  • Hoffman D. D. Singh M. & Prakash C. (2015). The interface theory of perception. Psychonomic bulletin & review 22(6) 1480-1506.

  • Hollerman J. & Schultz W. (1998). Dopamine Neruons Report an Error in the Temproal Prediction of Reward during Learning. Nature Neuroscience 1 304-309.

  • Huys Q. J. Eshel N. O’Nions E. Sheridan L. Dayan P. & Roiser J. P. (2012). Bonsai trees in your head: how the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Comput Biol 8(3) e1002410.

  • James W. (1884). II.—What is an emotion? Mind(34) 188-205.

  • James W. (1890). The Principles of Psychology. Cambridge MA: Harvard University Press.

  • Johnston Victor S. (1999). Why We Feel:The Science of Human Emotions. Reading MA: Perseus Books.

  • Kahneman D. (2003). Maps of bounded rationality: Psychology for behavioral economics. The American economic review 93(5) 1449-1475.

  • Kahneman D. (2011). Thinking Fast and Slow. New York: Farrar Straus and Giroux.

  • Kalis A. Kaiser S. & Mojzisch A. (2013). Why we should talk about option generation in decision-making research. Front. Psychol 4(555) 10.3389.

  • Keller L. R. & Ho J. L. (1988). Decision problem structuring: Generating options. Systems Man and Cybernetics IEEE Transactions on 18(5) 715-728.

  • Klein G. Wolf S. Militello L. & Zsambok C. (1995). Characteristics of skilled option generation in chess. Organizational Behavior and Human Decision Processes 62(1) 63-69.

  • Kringelbach M. L. & Berridge K. C. (2009). Towards a functional neuroanatomy of pleasure and happiness. Trends in cognitive sciences 13(11) 479-487.

  • Laird John E. J E. Newell A. & Rosenbloom Paul S. P. S. (1987). SOAR: An Architecture for General Intelligence. Artificial Intelligence 33 1–64.

  • Lang P. J. & Davis M. (2006). Emotion motivation and the brain: Reflex foundations in animal and human research. In G. E. M. J. J. K. S. Anders & D. Wildgruber (Eds.) Progress in Brain Research (Vol. Volume 156 pp. 3-29): Elsevier.

  • Lazarus R. (1991). Emotion and adaptation. New York: Oxford University Press.

  • LeDoux J. E. (2006). Emotional Memory: In Search of Systems and Synapsesa. Annals of the New York Academy of Sciences 702(1) 149-157.

  • Lee-Johnson C. P. & Carnegie D. A. (2009). Robotic Emotions: Navigation with Feeling. In J. Vallverdú & D. Casacuberta (Eds.) Handbook of Research on Synthetic Emotions and Sociable Robotics (pp. 88-117): IGI Global.

  • Liddell B. J. Brown K. J. Kemp A. H. Barton M. J. Das P. Peduto A. . . . Williams L. M. (2005). A direct brainstem–ìamygdala–cortical ‘alarm’ system for subliminal signals of fear. NeuroImage 24(1) 235-243.

  • Lucantonio F. Stalnaker T. A. Shaham Y. Niv Y. & Schoenbaum G. (2012). The impact of orbitofrontal dysfunction on cocaine addiction. Nature Neuroscience 15(3) 358-366.

  • MacDonald K. (2008). Effortful Control Explicit Processing and the Regulation of Human Evolved Predispositions. Psychological Review 115(4) 012–1031.

  • Madl T. Baars B. J. & Franklin S. (2011). The Timing of the Cognitive Cycle. PLoS ONE 6(4) e14803.

  • Madl T. & Franklin S. (2012). A LIDA-based Model of the Attentional Blink. Proceedings of the 11th International Conference on Cognitive Modelling 283-288.

  • Madl T. Franklin S. Chen K. & Trappl R. (2013). Spatial Working Memory in the LIDA Cognitive Architecture. In R. West & T. Stewart (Eds.) Proceedings of the 12th International Conference on Cognitive Modelling (pp. 384-390). Ottawa Canada: Carleton University.

  • Maes P. (1989). How to do the right thing. Connection Science 1 291–323.

  • Marieb E. N. & Hoehn K. (2007). Human Anatomy & Physiology (Seventh ed.). San Francisco CA: Pearson Benjamin Cummings.

  • Marinier R. & Laird J. E. (2008). Emotion-driven reinforcement learning. Cognitive science 115-120.

  • Marinier R. P. Laird J. E. & Lewis R. L. (2009). A computational unification of cognitive behavior and emotion. Cognitive Systems Research 10(1) 48-69.

  • McCall R. Franklin S. & Friedlander D. (2010). Grounded Event-Based and Modal Representations for Objects Relations Beliefs Etc. Paper presented at the FLAIRS-23 Daytona Beach FL.

  • McCall R. J. (2014). Fundamental motivation and perception for a systems-level cognitive architecture. The University of Memphis.

  • Mnih V. Kavukcuoglu K. Silver D. Rusu A. A. Veness J. Bellemare M. G. ... & Petersen S. (2015). Human-level control through deep reinforcement learning. Nature518(7540) 529.

  • Montague P. R. Dayan P. & Sejnowski T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. The Journal of neuroscience 16(5) 1936-1947.

  • Negatu A. (2006). Cognitively Inspired Decision Making for Software Agents: Integrated Mechanisms for Action Selection Expectation Automatization and Non-Routine Problem Solving: Ph.D. Dissertation The University of Memphis Memphis TN USA.

  • Neisser U. (1976). Cognition and Reality: Principles and Implications of Cognitive Psychology San Francisco: W. H. Freeman.

  • O’Doherty J. P. Dayan P. Friston K. Critchley H. & Dolan R. J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron 38(2) 329-337.

  • Pasquereau B. Nadjar A. Arkadir D. Bezard E. Goillandeau M. Bioulac B. . . . Boraud T. (2007). Shaping of motor responses by incentive values through the basal ganglia. Journal of Neuroscience 27 1176-1183.

  • Phelps E. A. (2006). Emotion and Cognition: Insights from Studies of the Human Amygdala. Annual Review of Psychology 57(1) 27-53. doi: doi:10.1146/annurev.psych.56.091103.070234

  • Picard R. (1997). Affective Computing. Cambridge MA: The MIT Press.

  • Picard R. W. (2003). Affective computing: challenges. International Journal of Human-Computer Studies 59(1‚Äì2) 55-64. doi: 10.1016/s1071-5819(03)00052-1

  • Purves D. Brannon E. M. Cabeza R. Huettel S. A. LaBar K. S. Platt M. L. & Woldorff M. G. (2008). Principles of cognitive neuroscience (Vol. 83): Sinauer Associates Sunderland MA.

  • Raab M. de Oliveira R. F. & Heinen T. (2009). How do people perceive and generate options? Progress in brain research 174 49-59.

  • Richard J. M. & Berridge K. C. (2011). Nucleus accumbens dopamine/glutamate interaction switches modes to generate desire versus dread: D1 alone for appetitive eating but D1 and D2 together for fear. The Journal of neuroscience 31(36) 12866-12879.

  • Roseman I. J. & Smith C. A. (2001). Appraisal theory: Overview assumptions varieties controversies Appraisal processes in emotion: Theory methods research (pp. 3-19). New York: Oxford University Press.

  • Rowe J. Hughes L. Eckstein D. & Owen A. M. (2008). Rule-Selection and Action-Selection have a Shared Neuroanatomical Basis in the Human Prefrontal and Parietal Cortex. Cerebral Cortex 18 2275-2285. doi: 10.1093/cercor/bhm249

  • Schoenbaum G. Takahashi Y. Liu T. L. & McDannald M. A. (2011). Does the orbitofrontal cortex signal value? Annals of the New York Academy of Sciences 1239(1) 87-99.

  • Shin Y. K. Proctor R. W. & Capaldi E. J. (2010). A review of contemporary ideomotor theory. Psychological Bulletin 136(6) 943-974. doi: 10.1037/a0020541

  • Sloman A. (1998). Damasio Descartes Alarms and Meta-management Proceedings Symposiumon Cognitive Agents: Modeling Human Cognition. San Diego: IEEE.

  • Sloman A. (1999). What Sort of Architecture is Required for a Human-like Agent? In M. Wooldridge & A. S. Rao (Eds.) Foundations of Rational Agency (pp. 35–52). Dordrecht Netherlands: Kluwer Academic Publishers.

  • Sloman A. & Croucher M. (1981). Why robots will have emotions.

  • Smith K. S. Berridge K. C. & Aldridge J. W. (2011). Disentangling pleasure from incentive salience and learning signals in brain reward circuitry. Proceedings of the National Academy of Sciences 108(27) E255-E264.

  • Snaider J. McCall R. & Franklin S. (2011). The LIDA Framework as a General Tool for AGI. Paper presented at the Artificial General Intelligence (AGI-11) Mountain View CA.

  • Squire L. R. & Kandel E. R. (2000). Memory: From mind to molecules: Macmillan.

  • Sun R. (2009). Motivational representations within a computational cognitive architecture. Cognitive Computation 1(1) 91-103.

  • Sutton R. S. & Barto A. G. (1998). Reinforcement Learning: An Introduction. Cambridge MA: MIT Press.

  • Thompson R. F. & Madigan S. A. (2007). Memory. Princeton NJ: Princeton University Press.

  • Tindell A. J. Smith K. S. Berridge K. C. & Aldridge J. W. (2009). Dynamic computation of incentive salience: ‘wanting’ what was never ‘liked’. The Journal of Neuroscience 29(39) 12220-12228.

  • Toates F. M. (1986). Motivational systems: CUP Archive.

  • Ward P. Suss J. Eccles D. W. Williams A. M. & Harris K. R. (2011). Skill-based differences in option generation in a complex task: A verbal protocol analysis. Cognitive processing 12(3) 289-300.

  • Watkins C. J. C. H. (1989). Learning from Delayed Rewards. Ph.D. thesis Cambridge University.

  • Westen D. (1999). The Scientific Status of Unconscious Processes: Is Freud Really Dead? Journal of the American Psychoanalytic Association 47(4) 1061-1106. doi: 10.1177/000306519904700404

  • Wimmer G. E. & Shohamy D. (2012). Preference by Association: How Memory Mechanisms in the Hippocampus Bias Decisions. Science 338(6104) 270-273. doi: 10.1126/science.1223252

  • Yerkes R. M. & Dodson J. D. (1908). The Relationship of Strength of Stimulus to Rapidity of Habit Formation. Journal of Comparative Neurology and Psychology 18 459–482.

  • Zacks J. M. Speer N. K. Swallow K. M. Braver T. S. & Reynolds J. R. (2007). Event Perception: A Mind–Brain Perspective. Psychological Bulletin 133(2) 273–293.

  • Zacks J. M. & Tversky B. (2001). Event structure in perception and conception. Psychological bulletin 127(1) 3.

Search
Journal information
Metrics
All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 57 57 57
PDF Downloads 52 52 52