基于深度强化学习的无人机栖落机动控制策略设计

Design of UAV Perching Maneuver Control Strategy Based on Deep Reinforcement Learning

关键词:  栖落机动  深度强化学习  固定翼无人机  神经网络

Key words:  Perching maneuver  Deep reinforcement learning  Fixed-wing UAV  Neural network

摘要:

无人机栖落机动飞行是一种无需跑道的降落方法，能够提升无人机在复杂环境下执行任务的适应能力。针对具有高非线性、多约束特性的无人机栖落机动过程，提出了一种基于模仿深度强化学习的控制策略设计方法。首先，建立了固定翼无人机栖落机动的纵向非线性动力学模型，并设计了无人机栖落机动的强化学习环境。其次，针对栖落机动状态动作空间大的特点，为了提高探索效率，通过模仿专家经验的方法对系统进行预训练。然后，以模仿学习得到的权重为基础，采用近端策略优化方法学习构建无人机栖落机动的神经网络控制器。最后，通过仿真验证了上述控制策略设计方法的有效性。

Abstract:

UAV perching maneuvering is a landing method that does not require a runway, which can improve the adaptability of UAV to perform tasks in complex environments. Aiming at the UAV perching maneuver process with high nonlinearity and multi-constraint characteristics, a control strategy design method based on imitating deep reinforcement learning is proposed. Firstly, a longitudinal nonlinear dynamic model of fixed-wing UAV perching maneuver is established, and a reinforcement learning environment for UAV perching maneuver is designed. Secondly, in view of the large action and state space of the perching maneuver, to improve the exploration efficiency, the system is pre-trained by imitating the experience of experts. Then, based on the weights obtained by imitation learning, the proximal policy optimization method is used to learn to build a neural network controller for UAV perching maneuver. Finally, simulations verify the effectiveness of the control strategy design method.