Learning Fast Quadruped Robot Gaits with the RL PoWER Spline Parameterization

Open access


Legged robots are uniquely privileged over their wheeled counterparts in their potential to access rugged terrain. However, designing walking gaits by hand for legged robots is a difficult and time-consuming process, so we seek algorithms for learning such gaits to automatically using real world experimentation. Numerous previous studies have examined a variety of algorithms for learning gaits, using an assortment of different robots. It is often difficult to compare the algorithmic results from one study to the next, because the conditions and robots used vary. With this in mind, we have used an open-source, 3D printed quadruped robot called QuadraTot, so the results may be verified, and hopefully improved upon, by any group so desiring. Because many robots do not have accurate simulators, we test gait-learning algorithms entirely on the physical robot. Previous studies using the QuadraTot have compared parameterized splines, the HyperNEAT generative encoding and genetic algorithm. Among these, the research on the genetic algorithm was conducted by (G l e t t e et al., 2012) in a simulator and tested on a real robot. Here we compare these results to an algorithm called Policy learning by Weighting Exploration with the Returns, or RL PoWER. We report that this algorithm has learned the fastest gait through only physical experiments yet reported in the literature, 16.3% faster than reported for HyperNEAT. In addition, the learned gaits are less taxing on the robot and more repeatable than previous record-breaking gaits.

1. Bongard, J., V. Zykov, H. Lipson. Resilient Machines Through Continuous Self-Modeling. - Science, Vol. 314, 2006, No 5802, 1118-1121.

2. Chernova, S., M. Veloso. An Evolutionary Approach to Gait Learning for Four-Legged Robots. - In: Intelligent Robots and Systems, 2004. (IROS’2004). Proceedings. 2004 IEEE/RSJ International Conference on, Vol. 3, 2004, 2562-2567.

3. Clune, J., B. Beckmann, C. Ofria, R. Pennock. Evolving Coordinated Quadruped Gaits with the Hyperneat Generative Encoding. - In: Evolutionary Computation, 2009. CEC’09. IEEE Congress on, 2009, 2764-2771.

4. Clune, J., K. Stanley, R. Pennock, C. Ofria. On the Performance of Indirect Encoding across the Continuum of Regularity. Evolutionary Computation. - IEEE Transactions, Vol. 15, 2011, No 3, 346-367.

6. Glette, K., G. Klaus, J. C. Zagal, J. Torresen. Evolution of Locomotion in a Simulated Quadruped Robot and Transferral to Reality. - In: Proceedings of the 17th International Symposium on Artificial Life and Robotics, 2012.

7. Hornby, G., S. Takamura, T. Yamamoto, M. Fujita. Autonomous Evolution of Dynamic Gaits with Two Quadruped Robots. - Robotics, IEEE Transactions on, Vol. 21, 2005, No 3, 402-410.

8. Kober, J., J. Peters. Learning Motor Primitives for Robotics. - In: Robotics and Automation, 2009. ICRA’09. IEEE International Conference on, 2009, 2112-2118.

9. Kormushev, P., B. Ugurlu, S. Calinon, N. Tsagarakis, D. Caldwell. Bipedal Walking Energy Minimization by Reinforcement Learning with Evolving Policy Parameterization. - In: Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on, 2011, 318-324.

10. Te’llez, R., C. Angulo, D. Pardo. Evolving the Walking Behaviour of a 12 Dof Quadruped using a Distributed Neural Architecture. - Biologically Inspired Approaches to Advanced Information Technology, 2006, 5-19.

11. Valsalam, V., R. Miikkulainen. Modular Neuroevolution for Multilegged Locomotion. - In: Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation, ACM, 2008, 265-272.

12. Yosinski, J., J. Clune, D. Hidalgo, S. Nguyen, J. C. Zagal, H. Lipson. Evolving Robot Gaits in Hardware: the Hyperneat Generative Encoding Vs. Parameter Optimization. - In: Proceedings of the 20th European Conference on Artificial Life, 2011.

13. Zykov, V., J. Bongard, H. Lipson. Evolving Dynamic Gaits on a Physical Robot. - In: Proceedings of Genetic and Evolutionary Computation Conference, Late Breaking Paper, GECCO, Vol. 4, 2004.

Cybernetics and Information Technologies

The Journal of Institute of Information and Communication Technologies of Bulgarian Academy of Sciences

Journal Information

CiteScore 2018: 0.84

SCImago Journal Rank (SJR) 2018: 0.215
Source Normalized Impact per Paper (SNIP) 2018: 0.595

Mathematical Citation Quotient (MCQ) 2017: 0.01

Cited By


All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 198 141 7
PDF Downloads 87 68 5