Sigma (Σ) is a cognitive architecture and system whose development is driven by a combination of four desiderata: grand unification, generic cognition, functional elegance, and sufficient efficiency. Work towards these desiderata is guided by the graphical architecture hypothesis, that key to progress on them is combining what has been learned from over three decades’ worth of separate work on cognitive architectures and graphical models. In this article, these four desiderata are motivated and explained, and then combined with the graphical architecture hypothesis to yield a rationale for the development of Sigma. The current state of the cognitive architecture is then introduced in detail, along with the graphical architecture that sits below it and implements it. Progress in extending Sigma beyond these architectures and towards a full cognitive system is then detailed in terms of both a systematic set of higher level cognitive idioms that have been developed and several virtual humans that are built from combinations of these idioms. Sigma as a whole is then analyzed in terms of how well the progress to date satisfies the desiderata. This article thus provides the first full motivation, presentation and analysis of Sigma, along with a diversity of more specific results that have been generated during its development.
If the inline PDF is not rendering correctly, you can download the PDF file here.
Anderson, J. R. 1983. The Architecture of Cognition. Cambridge, MA: Harvard University Press.
Anderson, J. R. 1990. The Adaptive Character of Thought. Hillsdale, NJ: Lawrence Erlbaum Associates.
Anderson, J. R. 2002. Spanning seven orders of magnitude: A challenge for cognitive modeling. Cognitive Science. 26: 85-112.
Anderson, J. R. 2007. How Can the Human Mind Occur in the Physical Universe. Oxford: Oxford University Press.
Anderson, J. R.; Bothell, D.; Byrne, M. D.; Douglass, S., Lebiere, C.; and Qi, Y. 2004. An integrated theory of the mind. Psychological Review. 111: 1036-1060.
Badler, N. I. 1997. Real-time virtual humans. In Proceedings of the IEEE Workshop on Non-Rigid and Articulated Motion, 28-36.
Bach, J. 2015. Modeling motivation in MicroPsi 2. In Proceedings of the 8th Conference on Artificial General Intelligence, 3-13.
Bailey, T.; and Durrant-Whyte, H. 2006. Simultaneous localisation and mapping (SLAM): Part II State of the art. Robotics and Automation Magazine. 13: 108–117.
Bell, C. G.; and Newell, A. 1971. Computer Structures: Readings and Examples. New York, NY: McGraw-Hill.
Bengio, Y.; Ducharme, R.; Vincent, P.; and Janvin, C. 2003. A neural probabilistic language model. The Journal of Machine Learning Research. 3: 1137-1155.
Best, B.; Lebiere, C.; and Scarpinatto, C. 2002. A model of synthetic opponents in MOUT training simulations using the ACT-R cognitive architecture. In Proceedings of the Eleventh Conference on Computer Generated Forces and Behavior Representation.
Bonasso, R. P.; Firby, R. J.; Gat, E.; Kortenkamp, D.; Miller, D. P.; and Slack, M. G. 1997. Experiences with an Architecture for Intelligent Reactive Agent. Journal of Experimental and Theoretical Artificial Intelligence. 9: 237-256.
Bostrom, N. 2001. Are you living in a computer simulation? Philosophical Quarterly. 53: 243-255.
Bridewell, W.; and Langley, P. 2011. A computational account of everyday abductive inference. In Proceedings of the Thirty-Third Annual Meeting of the Cognitive Science Society, 2289-2294.
Bubic, A.; von Cramon, D. Y.; and Schubotz, R. I. 2010. Prediction, cognition and the brain. Frontiers in Human Neuroscience. 4: 25.
Campbell, J.; Core, M.; Artstein, R.; Armstrong, L.; Hartholt, A.; Wilson, C.; Georgila, K.; Morbini, F.; Haynes, E.; Gomboc, D.; Birch, M.; Bobrow, J.; Chad Lane, H.; Gerten, J.; Leuski, A.; Traum, D.; Trimmer, M.; DiNinni, R.; Bosack, M.; Jones, T.; Clark, R.E.; and Yates, K.A., 2011. Developing INOTS to Support Interpersonal Skills Practice. In Proceedings of the Thirty-second Annual IEEE Aerospace Conference, 1-14.
Card, S. K.; Moran, T.P.; and Newell, A. 1983. The Psychology of Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum Associates.
Cassimatis, N. 2002. Polyscheme: A Cognitive Architecture for Integrating Multiple Representation and Inference Schemes. Ph.D. diss., Media Laboratory, MIT, Cambridge, Mass.
Chater, N.; Oaksford, M. 1999. Ten years of the rational analysis of cognition. Trends in Cognitive Sciences. 3: 57–65.
Chen, J.; Demski, A.; Han, T.; Morency, L-P.; Pynadath, D.; Rafidi, N.; and Rosenbloom, P. S. 2011. Fusing symbolic and decision-theoretic problem solving + perception in a graphical cognitive architecture. In Proceedings of the 2nd International Conference on Biologically Inspired Cognitive Architectures, 64-72.
Collobert, R.; and Weston, J. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning, 160-167.
Coste-Manière, E.; and Simmons, R. G. 2000. Architecture, the backbone of robotic systems. In Proceedings of the International Conference on Robotics and Automation, 67-72.
Dancy, C. L.; Ritter, F. E.; and Berry, K. 2012. Towards adding a physiological substrate to ACT-R. In Proceedings of the 21st Conference on Behavior Representation in Modeling and Simulation, 78-85.
Deering, S. 1988. Watching the waist of the protocol hourglass. Keynote address at ICNP ‘98. de Kleer, J. 1986. An assumption-based TMS. Artificial Intelligence. 28:127–162.
Dempster, A.P.; Laird, N.M.; and Rubin, D.B. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. 39: 1–38.
Derbinsky, N.; Laird, J. E.; and Smith, B. 2010. Towards efficiently supporting large symbolic declarative memories. In Proceedings of the 10th International Conference on Cognitive Modeling, 49-54.
Deutsch, D. 2011. The Beginning of Infinity: Explanations that Transform the World. London, UK: Penguin Books.
Domingos, P.; and Lowd, D. 2009. Markov Logic: An Interface Layer for Artificial Intelligence. San Raphael, CA: Morgan & Claypool.
Douglass, S.; Ball, J.; & Rodgers, S. 2009. Large declarative memories in ACT-R. In Proceedings of the 9th International Conference of Cognitive Modeling, 222-227.
Eliasmith, C. 2013. How to Build a Brain: A Neural Architecture for Biological Cognition. Oxford: Oxford University Press.
Falkenhainer, B; Forbus, K. D.; and Gentner, D 1989. The structure-mapping engine: Algorithm and examples. Artificial Intelligence. 41: 1–63.
Fikes, R., Hart, P; and Nilsson, N. 1972. Learning and Executing Generalized Robot Plans, Artificial Intelligence. 3: 251-288.
Forbus, K. D.; and Hinrichs, T. R. 2006. Companion Cognitive Systems: A step towards human-level AI. AI Magazine. 27:83-95.
Forgy, C. L. 1982. Rete: A fast algorithm for the many pattern/many object pattern match problem. Artificial Intelligence. 19: 17-37.
Frackowiak R; and Markram H. 2015. The future of human cerebral cartography: a novel approach. Philosophical Transactions of the Royal Society of London B: Biological Sciences. 370.
Frintrop, S.; Rome, E.; and Christensen, H.I. 2010: Computational visual attention systems and their cognitive foundation: A survey. ACM Transactions on Applied Perception. 7.
Garten, J.; Sagae, K.; Ustun, V.; and Dehghani, M. 2015. Combining distributed vector representations for words. In Proceedings of the NAACL Workshop on Vector Space Modeling for NLP, 95-101.
Goertzel, B. 2014. Artificial General Intelligence: Concept, State of the Art, and Future Prospects. Journal of Artificial General Intelligence. 5:1-46.
Goertzel, B.; Pennachin, C.; and Geisweiller, N. 2014. Engineering General Intelligence. Amsterdam: Atlantis Press.
Goodman, N. D.; Mansinghka, V. K.; Roy, D.; Bonawitz, K.; and Tenenbaum, J. B. 2008. Church: a language for generative models. In Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence, 220-229.
Hartholt, A.; Traum, D. Marsella, S. C.; Shapiro, A.; Stratou, G.; Leuski, A.; Morency, L.-P.; and Gratch, J. 2013. All together now: Introducing the Virtual Human Toolkit. In Proceedings of the 13th International Conference on Intelligent Virtual Agents, 368-381.
Hutter, M. 2005. Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability. Berlin: Springer-Verlag.
Itti, L.; and Baldi, P.F. 2006. Bayesian surprise attracts human attention. In Advances in Neural Information Processing Systems 18, 547-554.
Itti, L.; and Borji, A. 2013. State-of-the-art in visual attention modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence. 35: 185-207.
Jilk, D. J.; Lebiere, C.; O’Reilly, R. C. and Anderson, J. R. 2008. SAL: An explicitly pluralistic cognitive architecture. Journal of Experimental and Theoretical Artificial Intelligence. 20: 197-218.
Jones, M. N.; and Mewhort, D. J. 2007. Representing word meaning and order information in a composite holographic lexicon. Psychological review. 114: 1-37.
Jordan, M. I.; and Sejnowski, T. J. 2001. Graphical Models: Foundations of Neural Computation. Cambridge, MA: MIT Press.
Joshi, H.; Rosenbloom, P. S.; and Ustun, V. 2014. Isolated word recognition in the Sigma cognitive architecture. Biologically Inspired Cognitive Architectures. 10: 1-9.
Kahneman, D. 2011. Thinking Fast and Slow. New York, NY: Farrar, Straus and Giroux.
Kieras, D. E.; and Meyer, D. E. 1997. An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Human-Computer Interaction, 12: 391-438.
Koller, D.; and Friedman, N. 2009. Probabilistic Graphical Models: Principles and Techniques. Cambridge, MA: MIT Press.
Kopp, S.; Krenn, B.; Marsella, S.; Marshall, A. N.; Pelachaud C.; Pirker, H.; and Vilhjálmsson, H. 2006. Towards a common framework for multimodal generation: The behavior markup language. In Proceedings of the 6th International Conference on Intelligent Virtual Agents, 205-217.
Kschischang, F. R.; Frey, B. J.; and Loeliger, H.-A. 2001. Factor graphs and the sum-product algorithm. IEEE Transactions on Information Theory. 47: 498-519.
Laird, J. E. 2012. The Soar Cognitive Architecture. Cambridge, MA: MIT Press.
Laird, J E.; Newell, A.; and Rosenbloom, P. S. 1987. Soar: An Architecture for General Intelligence. Artificial Intelligence. 33: 1-64.
Laird, J. E.; and Rosenbloom, P. S. 1990. Integrating execution, planning, and learning in Soar for external environments, Proceedings of the Eighth National Conference on Artificial Intelligence,1022-1029.
Laird, J. E.; Rosenbloom, P. S.; and Newell, A. 1986. Chunking in Soar: The anatomy of a general learning mechanism. Machine Learning. 1: 11-46.
Langley, P.; and Choi, D. 2006. A unified cognitive architecture for physical agents. In Proceedings of the Twenty-First AAAI Conference on Artificial Intelligence, 1469-1474.
Langley, P.; Laird, J. E.; and Rogers, S. 2009. Cognitive architectures: Research issues and challenges. Cognitive Systems Research. 10: 141-160.
Lebiere, C. 2013. Summary presentation for panel on Consensus and Outstanding Issues. AAAI 2013 Fall Symposium on Integrated Cognition.
LeCun, Y.; Bengio, Y.; and Hinton, G. E. 2015. Deep Learning. Nature. 521: 436-444.
Lenat, D.; and Guha, R. V. 1990. Building Large Knowledge-Based Systems: Representation and Inference in the Cyc Project. Reading, MA: Addison-Wesley.
Madl, T.; and Franklin, S. 2012. A LIDA-based Model of the Attentional Blink. In Proceedings of the 11th International Conference on Cognitive Modeling, 283-288.
Maes, P.; and Nardi, D. eds. 1988. Meta-Level Architectures and Reflection. Amsterdam: North Holland.
Marr, D. 1982. Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. San Francisco, CA: W. H. Freeman.
Marsella, S.; and Gratch, J. 2009. EMA: A Process Model of Appraisal Dynamics. Journal of Cognitive Systems Research. 10: 70-90.
McCallum, A.; Rohanemanesh, K.; Wick, M.; Schultz, K.; and Singh, S. 2008. FACTORIE: Efficient probabilistic programming via imperative declarations of structure, inference and learning. In Proceedings of the NIPS workshop on Probabilistic Programming.
Meyer, D. E.; and Kieras, D. E. 1997. A computational theory of executive control processes and human multiple-task performance: Part 1. Basic Mechanisms. Psychological Review. 104: 3-65.
Mikolov, T.; Chen, K.; Corrado, G.; and Dean, J. 2013. Efficient estimation of word representations in vector space. In Proceedings of the International Conference on Learning Representations.
Milch, B.; Marthi, B.; Russell, S.; Sontag, D.; Ong, D. L.; and Kolobov, A. 2007. BLOG: Probabilistic models with unknown objects. In Introduction to Statistical Relational Learning eds. L. Getoor and B. Taskar. Cambridge, MA: MIT Press.
Mnih, A.; and Kavukcuoglu, K. 2013. Learning word embeddings efficiently with noise-contrastive estimation. In Advances in Neural Information Processing Systems, 2265-2273.
Minsky, M. 1986. The Society of Mind. New York, NY: Simon & Schuster.
Moors, A.; Ellsworth, P.C.; Scherer, K.R.; and Frijda, N.H. 2013. Appraisal theories of emotion: State of the art and future development. Emotion Review. 5: 119-124.
Murphy, K. 2002. Dynamic Bayesian Networks: Representation, Inference and Learning. Ph.D. diss., Computer Science Division, UC Berkeley, Berkeley, Calif.
Murphy, R. R. 2000. Introduction to AI Robotics. Cambridge, MA: MIT Press.
Newell, A. 1990. Unified Theories of Cognition. Cambridge, MA: Harvard University Press.
Newell, A.; Shaw, J. C.; and Simon, H. A. 1959. Report on a general problem-solving program. In Proceedings of the International Conference on Information Processing, 256-264.
Newell, A.; Yost, G. R.; Laird, J. E.; Rosenbloom, P. S.; and Altmann, E. 1991. Formulating the problem space computational model. In CMU Computer Science: A 25th Anniversary Commemorative ed. R. F. Rashid. New York, NY: ACM Press/Addison-Wesley.
Ng, A. Y.; and Russell, S. J. 2000. Algorithms for inverse reinforcement learning. In Proceedings of the 17th International Conference on Machine Learning, 663–670.
Niv, Y. 2009. Reinforcement learning in the brain. The Journal of Mathematical Psychology. 53: 139-154.
Oaksford, M.; and Chater, N. 2007. Bayesian Rationality: The Probabilistic Approach to Human Reasoning. Oxford: Oxford University Press.
O’Connor, T.; and Wong, H. Y. 2015. Emergent properties. In The Stanford Encyclopedia of Philosophy (Summer 2015 Edition), ed. E. N. Zalta.
O’Reilly, R. C.; and Munakata, Y. 2000. Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain. Cambridge, MA: MIT Press.
Ortony, A.; Norman, D. A.; and Revelle, W. 2005. Affect and Proto-affect in effective functioning. In Who Needs Emotions? The Brain Meets the Machine : eds. J. M. Fellous and M. A. Arbib. New York, NY: Oxford University Press.
Pearl, J. 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Francisco, CA: Morgan Kaufman.
Pynadath, D. V.; and Marsella, S. C. 2005. PsychSim: Modeling Theory of Mind with decision-theoretic agents. In Proceedings of the 19th International Joint Conference on Artificial Intelligence, 1181-1186.
Pynadath, D. V.; Rosenbloom, P. S.; and Marsella, S. C. 2014. Reinforcement learning for adaptive Theory of Mind in the Sigma cognitive architecture. In Proceedings of the 7th Annual Conference on Artificial General Intelligence, 143-154.
Pynadath, D. V.; Rosenbloom, P. S.; Marsella, S. C.; and Li, L. 2013. Modeling two-player games in the Sigma graphical cognitive architecture. In Proceedings of the 6th Conference on Artificial General Intelligence, 98-108. Berlin: Springer.
Rosenbloom, P. S. 1982. A world-championship-level Othello program. Artificial Intelligence. 19: 279-320.
Rosenbloom, P. S. 2006. A cognitive odyssey: From the power law of practice to a general learning mechanism and beyond. Tutorials in Quantitative Methods for Psychology. 2: 43-51.
Rosenbloom, P. S. 2009. Towards a new cognitive hourglass: Uniform implementation of cognitive architecture via factor graphs. In Proceedings of the 9th International Conference on Cognitive Modeling, 116-121.
Rosenbloom, P. S. 2010. Combining procedural and declarative knowledge in a graphical architecture. In Proceedings of the 10th International Conference on Cognitive Modeling, 205-210.
Rosenbloom, P. S. 2011a. Rethinking cognitive architecture via graphical models. Cognitive Systems Research. 12: 198-209.
Rosenbloom, P. S. 2011b. Mental imagery in a graphical cognitive architecture. In Proceedings of the 2nd International Conference on Biologically Inspired Cognitive Architectures, 314-323.
Rosenbloom, P. S. 2011c. From memory to problem solving: Mechanism reuse in a graphical cognitive architecture. In Proceedings of the 4th Conference on Artificial General Intelligence, 143-152.
Rosenbloom, P. S. 2012a. Deconstructing reinforcement learning in Sigma. In Proceedings of the 5th Conference on Artificial General Intelligence, 262-271. Berlin: Springer.
Rosenbloom, P. S. 2012b. Extending mental imagery in Sigma. In Proceedings of the 5th Conference on Artificial General Intelligence, 272-281.
Rosenbloom, P. S. 2012c. Towards a 50 msec cognitive cycle in a graphical architecture. In Proceedings of the 11th International Conference on Cognitive Modeling, 305-310.
Rosenbloom, P. S. 2014. Deconstructing episodic learning and memory in Sigma. In Proceedings of the 36th Annual Conference of the Cognitive Science Society, 1317-1322.
Rosenbloom, P. S. 2015. Supraarchitectural capability integration: From Soar to Sigma. In Proceedings of the 13th International Conference on Cognitive Modeling, 67-68.
Rosenbloom, P. S.; Demski, A.; Han, T.; and Ustun, V. 2013. Learning via gradient descent in Sigma. In Proceedings of the 12th International Conference on Cognitive Modeling, 35-40.
Rosenbloom, P. S.; Demski, A.; and Ustun, V. 2015. Efficient message computation in Sigma’s graphical architecture. Biologically Inspired Cognitive Architectures. 11: 1-9.
Rosenbloom, P. S.; Demski, A.; and Ustun, V. 2016. Rethinking Sigma’s graphical architecture while extending it to neural networks. In Proceedings of the 9th Conference on Artificial General Intelligence.
Rosenbloom, P. S.; Gratch, J.; and Ustun, V. 2015. Towards emotion in Sigma: From Appraisal to Attention. In Proceedings of the 8th Conference on Artificial General Intelligence, 142-151.
Rosenbloom, P. S.; Laird, J. E.; and Newell, A. 1988. Meta-levels in Soar. In Meta-Level Architectures and Reflection eds. P. Maes and D. Nardi. Amsterdam, Netherlands: North Holland.
Rosenbloom, P. S.; Lee, S.; and Unruh, A. 1990. Responding to impasses in memory-driven behavior: A framework for planning, Proceedings of the Workshop on Innovative Approaches to Planning, Scheduling, and Control, 181-191.
Rosenbloom, P. S.; Laird, J. E.; and Newell, A. 1987. Knowledge level learning in Soar. In Proceedings of Sixth National Conference on Artificial Intelligence, 499-504.
Rosenbloom, P. S.; Laird, J. E.; and Newell, A. eds. 1993. The Soar Papers: Research on Integrated Intelligence. Cambridge, MA: MIT Press.
Rumelhart, D. E.; Hinton, G. E.; and Williams, R. J. 1986. Learning representations by back-propagating errors. Nature. 323: 533-536.
Russell, S.; Binder, J.; Koller, D.; and Kanazawa, K. 1995. Local learning in probabilistic networks with hidden variables. In Proceedings of the 14th International Joint Conference on AI, 1146-1152.
Schneider, W.; and Shiffrin, R. M. 1977. Controlled and automatic human information processing: I. Detection, search, and attention. Psychological Review. 84: 1-66.
Shapiro, A. 2011. Building a character animation system. In Proceedings of the 4th International Conference on Motion in Games, 98-109.
Simon, H. A. 1956. Rational choice and the structure of the environment. Psychological Review. 63: 129–138.
Singla, P.; and Domingos, P. 2008. Lifted first-order belief propagation. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, 1094-1099.
Sun, R. 2006. The CLARION cognitive architecture: Extending cognitive modeling to social simulation In Cognition and Multi-Agent Interaction ed. R. Sun. New York, NY: Cambridge University Press.
Sun, R.; and Wilson, N. 2010. Motivational processes within the perception-action cycle. Perception-Action Cycle: Models, Architectures, and Hardware. New York, NY: Springer.
Sutton, R. S.; and Barto, A. G. 1998. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.
Swartout, W. 2010. Lessons learned from virtual humans. AI Magazine. 31: 9-20.
Tambe, M.; and Rosenbloom, P. S. 1994. Investigating production system representations for non-combinatorial match. Artificial Intelligence. 68: 155-199.
Turney, P. D.; and Pantel, P. 2010. From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research. 37: 141-188.
Ustun, V; and Rosenbloom, P. S. 2015. Towards adaptive, interactive virtual humans in Sigma. In Proceedings of the 15th International Conference on Intelligent Virtual Agents, 98-108.
Ustun, V.; Rosenbloom, P. S.; Kim, J.; and Li, L. 2015. Building high fidelity human behavior models in the Sigma cognitive architecture. In Proceedings of the 2015 Winter Simulation Conference, 3124-3125.
Ustun, V.; Rosenbloom, P. S.; Sagae, K.; and Demski, A. 2014. Distributed vector representations of words in the Sigma cognitive architecture. In Proceedings of the 7th Annual Conference on Artificial General Intelligence, 196-207.
Veloso, M. M.; and Carbonell, J. G. 1993. Derivational analogy in Prodigy: Automating case acquisition, storage, and utilization. Machine Learning. 10: 249–278.
Veness, J.; Ng, K. S.; Hutter, M.; Uther, W.; Silver, D. 2011. A Monte-Carlo AIXI approximation. Journal of Artificial Intelligence Research. 40: 95-142
Vere, S. and Bickmore, T. 1990. A Basic Agent. Computational Intelligence. 6: 41-60.
Wang, P. 2007. The logic of intelligence. In Artificial General Intelligence eds. B. Goertzel and C. Pennachin. New York, NY: Springer.
Whiten, A. ed. 1991. Natural Theories of Mind. Oxford: Basil Blackwell.