Deep Reinforcement Learning Approach for UAV Search Path Planning In Discrete Time and Space

Path planning for search missions carried out by Unmanned Aerial Vehicles (UAVs) is a challenging problem. This is due to UAV limited energy budget and the importance of time for search operations. The objective of this study is to come up with an approach to minimize the total search time required to locate a specific target. To achieve this, we deployed a deep reinforcement learning (DRL) model based on the Proximal Policy Optimization (PPO) algorithm to solve the combinatorial optimization problem of UAV search path planning within a minimized search time. A smart reward formulation is designed to achieve the learning goal, fulfill the search requirement, and encourage the agent to select search paths that minimize search time. In addition, we employed Optuna hyperparameter optimization framework to systematically select optimal parameters for the PPO model. Most importantly, thanks to the state representation we considered, the model is generalized and adaptable to various search environments. The PPO model succeeds to compute an accurate search path to be followed by the UAV searcher. Results of the model are compared with results previously obtained with a linear program. We found that the PPO achieves almost the same expected search time, which proves the great relevance of the reward design and the hyperparameters selection we made.