Краткое описание

Modular robotic systems are a group of identical robots capable of reconfiguring depending on the assigned tasks. The task of reconfiguring the kinematic structure is the construction of a trajectory connecting the initial and target positions for each module of the system. This paper presents a method for reconfiguring the kinematic structure of a modular robotic system using deep reinforcement learning. This method is based on the deep Q-learning algorithm. In addition, a method for the formation of Q-tables has been developed, which makes it possible to effectively scale the system, as well as to obtain the most complete information about the joint position of the system and transfer it to the neural network, a mechanism for forming the observation vector has been developed. The aim of the training is to build a reward-maximizing control algorithm. To assess the effectiveness of the proposed method, a computer simulation of a modular robotic system consisting of 5, 10 and 15 modules was created. The percentage of test episodes completed without collision was about 94% at 5 modules, 89% at 10 and 82% at 15, and the percentage of successfully completed episodes was about 87%, 64% and 0%, respectively. In addition, in most of the episodes that were completed without a collision, most of the modules were in target positions. In this connection, we can conclude that for the agent was successfully trained to reconfigure 5 and 10 modules, and for 15 a partial solution of the reconfiguration problem was obtained.