<abstract xmlns="http://www.w3.org/1999/xhtml"><p>We consider the problem of optimization in policy space for reinforcement learning. While a plethora of methods have been applied to this problem, only a narrow category of them proved feasible in robotics. We consider the peculiar characteristics of reinforcement learning in robotics, and devise a combination of two algorithms from the literature of derivative-free optimization. The proposed combination is well suited for robotics, as it involves both off-line learning in simulation and on-line learning in the real environment. We demonstrate our approach on a real-world task, where an Autonomous Underwater Vehicle has to survey a target area under potentially unknown environment conditions. We start from a given controller, which can perform the task under foreseeable conditions, and make it adaptive to the actual environment.</p></abstract>

We consider the problem of optimization in policy space for reinforcement learning. While a plethora of methods have been applied to this problem, only a narrow category of them proved feasible in robotics. We consider the peculiar characteristics of reinforcement learning in robotics, and devise a combination of two algorithms from the literature of derivative-free optimization. The proposed combination is well suited for robotics, as it involves both off-line learning in simulation and on-line learning in the real environment. We demonstrate our approach on a real-world task, where an Autonomous Underwater Vehicle has to survey a target area under potentially unknown environment conditions. We start from a given controller, which can perform the task under foreseeable conditions, and make it adaptive to the actual environment.

Combining Local and Global Direct Derivative-Free Optimization for Reinforcement Learning

Department of Advanced Robotics, Istituto Italiano di Tecnologia, via Morego, 30, 16163 Genova

Department of Computer, Control, and Management Engineering, Sapienza University of Rome, via Ariosto 25, 00185, Roma

Cybernetics and Information Technologies

We consider the problem of optimization in policy space for reinforcement learning. While a plethora of methods have been applied to this problem, only a...

Combining Local and Global Direct Derivative-Free Optimization for Reinforcement Learning

Published Online: Mar 22, 2013

Page range: 53 - 65

DOI: https://doi.org/10.2478/cait-2012-0021

Keywords
Reinforcement learning, policy search, derivative-free optimization, robotics, autonomous underwater vehicles

This content is open access.

Combining Local and Global Direct Derivative-Free Optimization for Reinforcement Learning

Published Online: Mar 22, 2013

Page range: 53 - 65

DOI: https://doi.org/10.2478/cait-2012-0021

KeywordsReinforcement learning, policy search, derivative-free optimization, robotics, autonomous underwater vehicles

This content is open access.

Keywords
Reinforcement learning, policy search, derivative-free optimization, robotics, autonomous underwater vehicles