On Intelligent Placement Decision-Making Algorithms for Wireless Digital Twin Networks via Bandit Learning

In this work, we present a novel approach to address the problem of minimizing the total cost in wireless digital twin network (WDTN) through the design of an intelligent digital twins (DTs) placement algorithm. The total cost is defined as the weighted sum of latency and energy consumption, evaluating the importance of these two factors. The dynamics in the WDTN require an adaptive and flexible approach to perform DT placement over time. To efficiently tackle the non-convex and non-stationary optimization problem for DT placement, we develop a multi-armed bandit (MAB) driven algorithm. Initially, we remodel the original problem of DT placement as a variation of the budgeted-MAB problem to enhance the efficiency of resources utilization within a fixed budget. Subsequently, we exploit the contextual information about the DTs and edge servers to establish associations across different time slots, effectively addressing the non-stationary nature of the problem. To further make the DT placement decisions and reduce uncertainty of the placement strategy, we propose an extension of the upper confidence bound (UCB) strategy. This extension incorporates both the efficiency of resources utilization and contextual information of the WDTN while considering energy consumption, latency, and other relevant factors. Extensive and comprehensive numerical simulations are conducted to evaluate the performance of our devised algorithm. The results demonstrate the superiority of the proposed method in terms of energy consumption, latency, and total cost when compared to various baseline schemes. These findings highlight the effectiveness and robustness of our developed algorithm in achieving significant improvements in cost reduction for WDTN.