In this paper, we propose a reinforcement learning world model that leverages the strengths of the state-of-the-art object-centric models. Our approach combines symbol-like object-centric representations, known as slots, with action representations to accurately predict the next state and reconstruct the current state of the environment. A key aspect of our method is the composition of actions and objects using an autoregressive transformer, which enables the model to efficiently capture the complex interactions between objects and actions in a given context. We present a comprehensive evaluation of our approach in various environments, demonstrating that our proposed method outperforms competing models. The source code of our model and training/testing scripts are publicly available at https://anonymous.4open.science/r/compas-1E03.
Download PDF at the IJCAI 2023 workshop website: https://nsa-wksp.github.io/assets/papers/COMPAS%20Compose%20Actions%20and%20Slots%20in%20Object-Centric%20World%20Models.pdf
Daniil Kirilenko, Vitaliy Vorobyov, Alexey Kovalev, Aleksandr Panov. COMPAS: Compose Actions and Slots in Object-Centric World Models // NSA: Neuro-Symbolic Agents Workshop. Thirty-Second International Joint Conference on Artificial Intelligence (IJCAI 2023). (Macao, S.A.R., 19–25 August 2023).