Reinforcement Learning Approach for Advanced Sleep Modes Management in 5G Networks

Advanced Sleep Modes (ASMs) correspond to a gradual deactivation of the Base Station (BS)'s components in order to reduce its Energy Consumption (EC). Different levels of Sleep Modes (SMs) can be considered according to the transition time (deactivation and activation durations) of each component. We propose in this paper a management solution for ASMs based on Q-learning approach. The target is to find the optimal durations for each SM level according to the requirements of the network operator in terms of EC reduction and delay constraints. The proposed solution shows that even with a high constraint on the delay, we can achieve high energy savings in a low load scenario (up to 57% of EC reduction) without inducing any impact on the delay. When the delay constraint is relaxed, we can achieve up to almost 90% of energy savings.