The paper considers the actual task of building a multimodal control system for an unmanned aerial vehicle of the quadrocopter type, integrating various methods of input, processing and transmission of information. The quadcopter accepts voice and gesture commands that are prepared and recognized through deep learning artificial neural networks. The general architecture of the system is given, its functions and software are described. The environment for flight simulation and visualization is considered. By means of the developed software package, an imitation of the flight of a quadrocopter was performed. Various teams and flight missions were tested, including flight over buildings. The conducted studies have shown that the unmanned aerial vehicle successfully recognizes commands, and the visualization system adequately reflects the process of the flight task.
Abramov, N., Talalaev, A., Fralenko, V., Khachumov, M. Multimodal control and visualization system for unmanned aerial flight // Aerospace Instrument-Making, № 9, pp. 3–11.