Deep Reinforcement Learning Aided Online Trajectory Optimization of Cellular-Connected UAVs with Offline Map Reconstruction

To reduce the outage of the connection between unmanned aerial vehicles (UAVs) and cellular networks in complex real-time channel state, and reduce the energy consumption of UAV during flight mission, an online trajectory optimization scheme of UAV based on outage probability knowledge map reconstruction is proposed. The outage probability knowledge map is a database that simulates the connection between UAV and the cellular network during real hovers. The UAV first samples sparsely from the target area and calculates the outage probability of the sampling point, and then uses the Kriging algorithm to reconstruct the outage probability knowledge map. Based on the reconstructed outage probability knowledge map, with the goal of minimizing the energy consumption of UAV task execution, the UAV trajectory optimization problem is established, and a trajectory optimization algorithm based on deep reinforcement learning (DRL) is proposed to solve it. Numerical results show that the proposed online trajectory optimization scheme based on outage probability knowledge map can obtain great returns in terms of maintaining connectivity, reducing task completion time and energy consumption.