Multi-Agent Reinforcement Learning for URLLC Modern Random Access

We tackle the problem of scheduling Ultra-Reliable Low-Latency Communications (URLLC) on the uplink by relying on Modern Random Access (MRA) and Multi-Agent Reinforcement Learning (MARL). We first formulate a URLLC MRA (uMRA) problem with a Decentralized Partially Observable Markov Decision Process (Dec-POMDP). We then propose uMRA-HAPPO, a MARL-based solution employing the Heterogeneous Agents Proximal Policy Optimization (HAPPO) algorithm. Finally, we assess the performance of our proposed algorithm in scenarios inspired by 3GPP use cases, showing higher reliability than deterministic and random traditional strategies under a strict latency constraint.