Deep Reinforcement Learning-Based Beam Selection and Tracking for Energy-Efficient mmWave Beamforming in 6G Networks

Abstract

Millimeter-wave (mmWave) communications are anticipated to be highly significant to the sixth generation (6G) wireless networks because of the ultra-high data rate provided by the mmWave communications. Yet, conventional mmWave bands have a high path loss and mobility sensitivity, which implies incessant and precise beam alignment, which is an immense feat in dynamic settings. In this paper, the framework of an energy-efficient beam selection and tracking framework in 6G mmWave systems based on deep reinforcement learning (DRL) is presented. The beam management process is expressed as a Markov Decision Process (MDP) and a Proximal Policy Optimization (PPO) agent is deployed to learn an optimal policy of controlling the beam in real-time. The proposed DRL agent uses the information of both channel states feedback and user mobility to choose adaptively the beam directions without the need of carrying out a beam search exhaustively or using a fixed codebook. An optimal reward function, which is specific to the environment of the robot, achieves a tradeoff of signal quality and energy consumption to deliver a confident beam alignment alongside minimal overheads. The simulations done extensively over a wide range of mobility conditions show that the PPO-based strategy gets up to 30 percent savings in energy and 95 percentage of beam alignment accuracy when compared to the existing schemes, exhaustive search and location-based beam-forming. All these findings confirm that intelligent beam management will be a viable method of increasing the energy-efficiency and reliability of 6G mmWave communications. The suggested framework provides the main reference to mobility-oriented scalable beamforming within the future of wireless networks.