Adaptive hybrid car following strategy using cooperative adaptive cruise control and deep reinforcement learning

The hybrid strategy can fully utilize the exploration ability of reinforcement learning while using rule-based strategies to ensure a lower performance limitation. The key challenge in this strategy is determining the optimal point for rule-based intervention within the learning-based framework,where a fixed empirical coefficient is usually applied to guide this intervention. However, the fixed coefficient is less effective in adapting to complex traffic conditions, particularly in mixed traffic flows, and significantly limits the overall performance. To overcome this, an adaptive hybrid car following strategy is proposed. Different from the fixed coefficient based strategy, two adaptive coefficients are calculated at each timestep. One adaptive coefficient is calculated iteratively using a Kalman Filter over multi-timestamp predictions, and another one is calculated through Monte Carlo Tree Search algorithm. These adaptive coefficients dynamically decide the optimal point, thereby enhancing the system's ability to handle complex scenarios such as mixed traffic flows. The effectiveness of the proposed algorithm has been validated through mathematical proof and extensive numerical simulations. Results from simulations, supported by both statistical analysis and specific case studies, demonstrate that, compared to the conventional method with a fixed coefficient, the proposed strategy significantly enhances safety, comfort, and efficiency in car following under complex traffic conditions.