<abstract xml:lang="en" xmlns="http://www.w3.org/1999/xhtml"><p>Cognitive Science and Artificial Intelligence share compatible goals of understanding and possibly generating broadly intelligent behavior. In order to determine if progress is made, it is essential to be able to evaluate the behavior of complex computational models, especially those built on general cognitive architectures, and compare it to benchmarks of intelligent behavior such as human performance. Significant methodological challenges arise, however, when trying to extend approaches used to compare model and human performance from tightly controlled laboratory tasks to complex tasks involving more open-ended behavior. This paper describes a model comparison challenge built around a dynamic control task, the Dynamic Stocks and Flows. We present and discuss distinct approaches to evaluating performance and comparing models. Lessons drawn from this challenge are discussed in light of the challenge of using cognitive architectures to achieve Artificial General Intelligence.</p></abstract>

Cognitive Science and Artificial Intelligence share compatible goals of understanding and possibly generating broadly intelligent behavior. In order to determine if progress is made, it is essential to be able to evaluate the behavior of complex computational models, especially those built on general cognitive architectures, and compare it to benchmarks of intelligent behavior such as human performance. Significant methodological challenges arise, however, when trying to extend approaches used to compare model and human performance from tightly controlled laboratory tasks to complex tasks involving more open-ended behavior. This paper describes a model comparison challenge built around a dynamic control task, the Dynamic Stocks and Flows. We present and discuss distinct approaches to evaluating performance and comparing models. Lessons drawn from this challenge are discussed in light of the challenge of using cognitive architectures to achieve Artificial General Intelligence.

Editorial: Cognitive Architectures, Model Comparison and AGI

Psychology Department, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213, USA

Dynamic Decision Making Laboratory, Social and Decision Sciences Department, Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213, USA

Alion Science and Technology, 4949 Pearl East Circle, Suite 200, Boulder, CO 80301, USA

Journal of Artificial General Intelligence

{"article-title":"Editorial: Cognitive Architectures, Model Comparison and AGI"}

Cognitive Science and Artificial Intelligence share compatible goals of understanding and possibly generating broadly intelligent behavior. In order to...

Editorial: Cognitive Architectures, Model Comparison and AGI

Published Online: Nov 23, 2011

Page range: 1 - 19

DOI: https://doi.org/10.2478/v10229-011-0006-4

Keywords
Cognitive Architectures, Model Comparison, Dynamic Stocks and Flows

This content is open access.

Editorial: Cognitive Architectures, Model Comparison and AGI

Published Online: Nov 23, 2011

Page range: 1 - 19

DOI: https://doi.org/10.2478/v10229-011-0006-4

KeywordsCognitive Architectures, Model Comparison, Dynamic Stocks and Flows

This content is open access.

Keywords
Cognitive Architectures, Model Comparison, Dynamic Stocks and Flows