Decentralized Multi-Agent Deep Reinforcement Learning Based Rate Selection of NOMA Assisted XP-HARQ

Non-orthogonal multiple access with cross-packet hybrid automatic repeat request (NOMA-XP-HARQ) is a promising solution for achieving ultra-reliable low-latency communications (URLLC) in Internet of Things (IoT) applications, primarily due to its flexibility in tuning reliability and latency through appropriate rate adaptation. This paper focuses on maximizing the long-term average throughput (LTAT) by selecting optimal rates while ensuring outage performance and throughput for each IoT device. However, the complex transmission mechanisms inherent in NOMA-XP-HARQ, coupled with outdated channel state information, impede optimal design using traditional optimization tools. To address this challenge, we develop a decentralized multi-agent deep reinforcement learning (MADRL) framework based on Markov game theory, which effectively tackles scalability issues and the non-stationarity of multi-device environments, in contrast to centralized single-agent deep reinforcement learning (DRL) approaches. Monte Carlo simulations demonstrate the superiority of the proposed MADRL algorithms over the centralized DRL approach.