Co-optimization of Morphology and Behavior of Modular Robots via Hierarchical Deep Reinforcement Learning

Jieqiang Sun
Jilin University
Meibao Yao
Jilin University
Xueming Xiao
Changchun University Of Science And Technology
Zhibing Xie
Jilin University
Bo Zheng
Shanghai Aerospace Control Technology Institute
Paper Website

Paper ID 96

Session 12. Robot Mechanisms & Control

Poster Session Thursday, July 13

Poster 32

Abstract: Modular robots hold the promise of changing their shape and even dimension to adapt to various tasks and environments. To realize this superiority, it is essential to find the appropriate morphology and its corresponding behavior simultaneously to ensure optimality of the reconfiguration. However, achieving co-optimization is challenging because robotic configuration and motion are interactive and coupled with each other, as well as their optimization processes. To this end, we proposed a co-optimization framework based on hierarchical Deep Reinforcement Learning (DRL), consisting of a configuration model and a motion model based on the Twin Delayed Deep Deterministic policy gradient algorithm (TD3). The two network models update asynchronously with a shared reward to ensure co-optimality. We conduct simulations and experiments with the Webots platform to validate the proposed framework, and the preliminary results show that it yields high quality optimization schemes and thus allows modular robots to be more adaptive to dynamic and multi-task scenarios.