Exploration for Understanding in Cognitive Modeling
The cognitive modeling and artificial general intelligence research communities may reap greater scientific return on research investments - may achieve an improved understanding of architectures and models - if there is more emphasis on systematic sensitivity and necessity analyses during model development, evaluation, and comparison. We demonstrate this methodological prescription with two of the models submitted for the Dynamic Stocks and Flows (DSF) Model Comparison Challenge, exploring the complex interactions among architectural mechanisms, knowledge-level strategy variants, and task conditions. To cope with the computational demands of these analyses we use a predictive analytics approach similar to regression trees, combined with parallelization on high performance computing clusters, to enable large scale, simultaneous search and exploration.
If the inline PDF is not rendering correctly, you can download the PDF file here.
Alexander W. P. and Grimshaw S. D. 1996. Treed regression. Journal of Computational and Graphical Statistics. 5: 156-175.
Anderson J. R. 2007. How can the human mind occur in the physical universe? Oxford University Press Oxford UK.
Bush R. and Mosteller F. 1955. Stochastic Models for Learning. John Wiley & Sons New York.
Dutt V. and Gonzalez C. 2007. Slope of inflow impacts dynamic decision making. In Proceedings of the 25th International Conference of the System Dynamics Society (pp. 79). Boston MA: System Dynamics Society.
Estes W. K. 2002. Traps in the route to models of memory and decision. Psychonomic Bulletin & Review 9(1) 3-25.
Friedman J. 1991. Multivariate adaptive regression splines. The Annals of Statistics. 19: 1-141.
Gluck K. A. Scheutz M. Gunzelmann G. Harris J. and Kershner J. 2007. Combinatorics meets processing power: Large-scale computational resources for BRIMS. In Proceedings of the Sixteenth Conference on Behavior Representation in Modeling and Simulation 73-83 Orlando FL: Simulation Interoperability Standards Organization.
Gonzalez C. and Dutt V. 2007. Learning to control a dynamic task: A system dynamics cognitive model of the slope effect. In Proceedings of the 8th International Conference on Cognitive Modeling 61-66. Ann Arbor MI.
Halbrügge M. in press. Keep it simple - A case study of model development in the context of the Dynamic Stock and Flows (DSF) task. Journal of Artificial General Intelligence.
Kase S.E. Ritter F.E. and Schoelles M. 2007. Using HPC and PGAs to optimize noisy computational models of cognition. In Innovations and Advanced Techniques in Systems Computing Sciences and Software Engineering.
Kintsch W. Healy A. F. Hegarty M. Pennington B. F. Salthouse T. 1999. Models of working memory: Eight questions and some general issues. In A. Miyake and P. Shah (Eds.) Models of working memory: Mechanisms of active maintenance and executive control. (pp. 412-441). New York: Cambridge University Press.
Knofcyznski G. T. and Mundfrom D. 2008. Sample sizes when using multiple linear regression for prediction. Educational and Psychological Measurement. 68:431-442.
Langley P. Laird J. E. and Rogers S. 2008. Cognitive architectures: Research issues and challenges. Cognitive Systems Research 10(2) 141-160.
McClelland J. L. 2009. The place of modeling in cognitive science. Topics in Cognitive Science 1 11-38.
Moore L. R. 2010. Cognitive Model Exploration and Optimization: A New Challenge for Computational Science. In T. Jastrzembski (Ed.) Proceedings of the 19th Behavior Representation in Modeling and Simulation (BRIMS) Conference. Charleston SC.
Moore L. R. Kopala M. Mielke T. Krusmark M. and Gluck K. A. (2010). Simultaneous Performance Exploration and Optimized Search with Volunteer Computing. In Proceedings of the ACM International Symposium on High Performance Distributed Computing (HPDC). Chicago IL.
Pew R. W. Gluck K. A. and Deutsch S. 2005. Accomplishments challenges and future directions for human behavior representation. In K. A. Gluck and R. W. Pew (Eds.) Modeling human behavior with integrated cognitive architectures: Comparison evaluation and validation (pp. 397-414). Mahwah NJ: Lawrence Erlbaum Associates.
Reitter D. In press. Metacognition and multiple strategies in a cognitive model of online control. Journal of Artificial General Intelligence.
Rescorla R. A. and Wagner A. R. 1972. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black and W. F. Prokasy (Eds.) Classical conditioning II. Current research and theory 64-99. New York: Appleton-Century-Crofts.
Wallach D. and Lebiere C. 2003. Conscious and unconscious knowledge: Mapping to the symbolic and subsymbolic levels of a hybrid architecture. In Jimenez L. (Ed.) Attention and Implicit Learning. Amsterdam Netherlands: John Benjamins Publishing Company.