Peijun Dong

The next generation of wireless network 6G is envisioned to enable seamless global connectivity for the Internet of Everything (IoE) on the Earth. To this end, implementing high-speed wireless communication from the deep ocean to the sea surface leveraging multiple autonomous underwater vehicles (AUVs) with underwater wireless optical communication (UWOC) is an emerging and promising technology that enables real-time data collection for accurate underwater exploration and monitoring, for example, coordinated moving target monitoring. However, multihop UWOC is more susceptible to beam misalignment and positional uncertainty caused by external interference in the harsh environment. To address these challenges, we design a cooperative movement scheme for multiple AUVs based on a deep reinforcement learning (DRL) approach to perform robust optical communication. We first model the optical channel and then analyze the link performance to satisfy the bit error rate (BER) requirements. Afterward, we map the cooperative optical communication problem into a Markov decision process (MDP) and then we propose a deep deterministic policy gradient (DDPG)-based cooperative movement strategy, which is integrated with the extended Kalman filter (EKF) technique. Finally, we design a multi-AUV adaptive adjustment scheme for enhanced optical link adaptation, including an optical link distance adjustment algorithm, and an adaptive transmit power adjustment algorithm based on twin-delayed deep deterministic policy gradient (TD3). Through extensive simulations, it is demonstrated that the proposed algorithms are effective in achieving cooperative and adaptive underwater optical communication via multi-AUV under mobile scenarios.

Integrating large language models (LLMs) into reinforcement learning (RL) promises to enhance the learning performance. Traditional RL faces challenges in industrial settings, including complex environments, safety concerns, and multimodal data. As powerful tools for contextual learning and reasoning, LLMs can address issues inherent in traditional RL. This paper introduces a generic LLM4RL framework, and investigates how LLM4RL can improve learning performance in autonomous driving.

In complex underwater environments, multiple autonomous underwater vehicles (AUVs) typically use underwater acoustic communication to collaboratively search for unknown targets. However, traditional underwater acoustic communication suffers from high latency and low bandwidth issues. In contrast, optical communication provides higher bandwidth and lower latency, but its propagation distance is limited, which seriously affects the efficiency of AUV search. To address this challenge, this paper proposes a multi-AUV acoustic-optical multimodal communication target search scheme based on deep reinforcement learning (DRL). Firstly, the AUV search problem under communication constraints is analyzed, aiming to maximize search success rate, minimize search time, and maximize effective search area per unit time. Secondly, this problem is modeled as a distributed partially observable Markov decision process (Dec-POMDP), and then a multi-AUV acoustic-optical multimodal search algorithm (AOSA) is designed based on the multi-agent deep deterministic policy gradient (MADDPG) method. The AOSA algorithm enhances information sharing among AUVs through acoustic-optical multimodal communication, promoting accurate updating of probability maps. Additionally, by incorporating pheromones to correction probability maps, it enables AUVs to effectively adjust their search paths, thereby improving the efficiency of collaborative search. The numerical simulation results verify the efficiency of this scheme.

Multiple autonomous underwater vehicles (AUVs) integrating multi-agent reinforcement learning (MARL) have made remarkable achievement and widely utilized for underwater search and rescue missions. However, to perform collaborative multi-AUV search efficiently in harsh and communication-constrained marine environments, challenging issues need to be addressed, such as cold-start problem and poor collaborative information fusion. To deal with these challenges, this paper proposes a MACS scheme which integrates the reasoning capabilities of large language models (LLMs) into the MARL framework to solve the cold-start problem and facilitate efficient collaborative information fusion. In MACS, to alleviate the cold-start problem of MARL caused by the lack of prior knowledge, we design a LEMACS algorithm, which leverages LLMs to infer the initial Target Probability Map (TPM) from search tasks and underwater terrain information to accelerate the search process. Furthermore, to address low efficient data exchange and fusion issue under unstable channel, we propose a LLM-enhanced link selection algorithm LESCL which integrates TPM information and AUV link metrics to optimize the link selection procedure to enhance multi-AUV cooperative search information fusion. To validate the effectiveness of the proposed algorithms, we conduct extensive numerical simulations using open-source regional underwater terrain data, such as coral reef map dataset of Arizona State University (ASU) and the terrain data of the Dongsha Islands, and the simulation results indicate that MACS achieves a search success rate of up to 95% in emergency multi-AUV cooperative search missions. The code is available at https://github.com/SDUST-smartocean/MACS.