How and why actions are selected: action selection and the dark room problem

Open access

Abstract

In this paper, I examine an evolutionary approach to the action selection problem and illustrate how it helps raise an objection to the predictive processing account. Clark examines the predictive processing account as a theory of brain function that aims to unify perception, action, and cognition, but - despite this aim - fails to consider action selection overtly. He off ers an account of action control with the implication that minimizing prediction error is an imperative of living organisms because, according to the predictive processing account, action is employed to fulfill expectations and reduce prediction error. One way in which this can be achieved is by seeking out the least stimulating environment and staying there (Friston et al. 2012: 2). Bayesian, neuroscientific, and machine learning approaches into a single framework whose overarching principle is the minimization of surprise (or, equivalently, the maximization of expectation. But, most living organisms do not find, and stay in, surprise free environments. This paper explores this objection, also called the “dark room problem”, and examines Clark’s response to the problem. Finally, I recommend that if supplemented with an account of action selection, Clark’s account will avoid the dark room problem.

Arias-Carrión, Ó. & Pöppel, E. 2007. Dopamine, learning, and reward-seeking behavior. Acta neurobiologiae experimentalis, 67 (4): 481-488.

Barron, A. B. Søvik, E. & Cornish, J. L. 2010. The roles of dopamine and related compounds in reward-seeking behavior across animal phyla. Frontiers in behavioral neuroscience, 4 (163).

Basso, M. A., & Wurtz, R. H. 1997. Modulation of neuronal activity by target uncertainty. Nature, 389 (6646), 66-69.

Biederman, I. 1987. Recognition-by-components: a theory of human image understanding. Psychological review, 94 (2), 115.

Brown, H., Friston, K. & Bestmann, S. 2011. Active inference, attention, and motor preparation. Frontiers in psychology, 2 (218).

Chapman, P. 1982. The Origins of Troglobites. Proceedings of the University of Bristol Spelaeological Society 16 (2): 133–141.

Clark, A. 2012. Dreaming the Whole Cat: Generative Models, Predictive Processing, and the Enactivist Conception of Perceptual Experience. Mind, 121 (483): 753-771.

Clark, A. 2013. Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences, 36 (3), 181-204.

Clark, A. 2014. Perceiving as Predicting. In Mohan, M., Biggs, S., & Stokes, D. (eds.). Perception and Its Modalities. New York: Oxford University Press.

Dickinson, A. & Balleine, B. 2002. The role of learning in the operation of motivational systems. In C. R. Gallistel. (ed). Steven’s handbook of experimental psychology: Learning, motivation and emotion. Volume 3. 3rd edition. New York: John Wiley & Sons.

Eliasmith, C. 2007. How to build a brain: From function to implementation. Synthese, 159 (3): 373-388.

Friston, K. 2003. Learning and inference in the brain. Neural Networks, 16 (9): 1325-1352.

Friston, K.J., Daunizeau, J., & Kiebal, S.J., 2009 Reinforcement Learning or Active Inference? PLoS ONE 4 (7): e6421: 1-13.

Friston, K. 2011. Embodied inference: Or I think therefore I am, if I am what I think. In Tschacher, W. & Bergomi, C. (eds.). The implications of embodiment: Cognition and communication. Exeter: Imprint Academic.

Friston, K. & Stephan, K. E. 2007. Free-energy and the brain. Synthese, 159 (3): 417-458.

Friston, K., Thornton, C. & Clark, A. 2012. Free-energy minimization and the dark-room problem. Frontiers in psychology 3 (130): 1-7.

Glimcher, P. W. 2011. Foundations of neuroeconomic analysis. New York: Oxford University Press.

Glimcher, P. W. & Sparks, D. L. 1992. Movement selection in advance of action in the superior colliculus. Nature, 355 (6360): 542-545.

Godfrey-Smith, P. 2002. Environmental complexity and the evolution of cognition. In Sternberg, R. & Kaufman, J. (eds.). The evolution of intelligence. Mahwah: Laurence Erlbaum Associates.

Hohwy, J. 2010. The hypothesis testing brain: Some philosophical applications. In Christensen W., Schier, E. & Sutton, J. (eds.). ASCS09: Proceedings of the 9th Conference of the Australasian Society for Cognitive Science. Sydney: Macquarie Centre for Cognitive Science.

Hohwy, J., Roepstorff, A. & Friston, K. 2008. Predictive coding explains binocular rivalry: an epistemological review. Cognition, 108 (3): 687-701.

Houston, A. I., McNamara, J. M. & Steer, M. D. 2007. Do we expect natural selection to produce rational behaviour? Philosophical Transactions of the Royal Society B: Biological Sciences, 362 (1485): 1531-1543.

Hubel, D. H. & Wiesel, T. N. 1965. Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat. Journal of neurophysiology, 28 (2): 229-289.

Huebner, B. 2012. Surprisal and valuation in the predictive brain. Frontiers in psychology, 3 (415).

Kable, J. W. & Glimcher, P. W. 2007. The neural correlates of subjective value during intertemporal choice. Nature neuroscience, 10 (12): 1625-1633.

Knill, D. C. & Pouget, A. 2004. The Bayesian brain: the role of uncertainty in neural coding and computation. TRENDS in Neurosciences, 27 (12): 712-719.

Lorenz, E. N. 1982. Atmospheric predictability experiments with a large numerical model. Tellus, 34(6): 505-513.

Maes, P. 1990. Situated agents can have goals. Robotics and autonomous systems, 6(1): 49-70.

McClure, S. M., Laibson, D. I., Loewenstein, G. & Cohen, J. D. 2004. Separate neural systems value immediate and delayed monetary rewards. Science, 306 (5695): 503-507.

Platt, M. L. & Glimcher, P. W. 1999. Neural correlates of decision variables in parietal cortex. Nature, 400(6741): 233-238.

Rauss, K., Schwartz, S. & Pourtois, G. 2011. Top-down effects on early visual processing in humans: a predictive coding framework. Neuroscience & Biobehavioral Reviews, 35(5): 1237-1253.

Riesenhuber, M. & Poggio, T. 2000. Models of object recognition. Nature neuroscience, 3: 1199-1204.

Schall, J. D. & Hanes, D. P. 1993. Neural basis of saccade target selection in frontal eye field during visual search. Nature, 366 (6454): 467-469.

Schultz, W. 1998. Predictive reward signal of dopamine neurons. Journal of neurophysiology, 80 (1): 1-27.

Shea, N. 2012. Reward Prediction Error Signals are Meta-Representational. Noûs, 48 (2): 314-341.

Sherrington, C. S. 1906. The Integrative Action of the Nervous System. Connecticut: Yale University Press.

Spurrett, D. 2012. What is to be Done? Why Reward is Difficult to Do Without. Frontiers in psychology, 3: 412.

Sterelny, K. 2003. Thought in a hostile world: The evolution of human cognition. Oxford: Blackwell Publishing Ltd.

Tinbergen, N. 1951. The study of instinct. Michigan: Clarendon Press.

Journal Information

Metrics

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 135 135 48
PDF Downloads 38 38 10