The application of learning-based control methods in robotics presents significant challenges. One is that model-free reinforcement learning algorithms use observation data with low sample efficiency. To address this challenge, a prevalent approach is model-based reinforcement learning, which involves employing an environment dynamics model. We suggest approximating transition dynamics with symbolic expressions, which are generated via symbolic regression. Approximation of a mechanical system with a symbolic model has fewer parameters than approximation with neural networks, which can potentially lead to higher accuracy and quality of extrapolation. We use a symbolic dynamics model to generate trajectories in model-based policy optimization to improve the sample efficiency of the learning algorithm. We evaluate our approach across various tasks within simulated environments. Our method demonstrates superior sample efficiency in these tasks compared to model-free and model-based baseline methods.
DOI: 10.48550/arXiv.2407.13518
Скачать PDF или читать онлайн на arXiv.org (англ.): https://arxiv.org/html/2407.13518v1
ResearchGate: https://www.researchgate.net/publication/382363535_Model-based_Policy_Optimization_using_Symbolic_World_Model
Andrey Gorodetskiy, Konstantin Mironov, Aleksandr Panov. Model-based Policy Optimization Using Symbolic World Model // The 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2024) (Abu Dhabi, UAE, 14–18 October 2024).