Hierarchical Safe MARL-CBF for Persistent Multi-Robot Sensing

Safe multi-agent reinforcement learning is particularly challenging when robots must persistently explore a dynamic environment using only partial observations and limited inter-robot communication. We study persistent multi-robot sensing under range-limited and bandwidth-constrained communication, where each robot observes only its local surroundings and exchanges information with nearby teammates over a time-varying communication graph. Our motivating application is smoke-field monitoring, where the field evolves according to unknown spatiotemporal dynamics and must be estimated online from distributed robot measurements using a Gaussian Process.

We formulate this problem as a constrained decentralized partially observable Markov decision process and develop a hierarchical safe MARL framework based on MAPPO. A high-level policy assigns sensing sub-goals using each robot's local observation, communicated teammate information, and the current field belief, including estimated smoke concentration, predictive uncertainty, and information age. Because measurements become stale as the field evolves, the objective is to maximize fresh coverage over time rather than perform one-time exploration. Centralized training allows the critic to exploit global information during learning, while decentralized execution requires each robot to act from locally available information and messages received within its communication constraints.

Safety is enforced at both levels of the decision hierarchy. Before execution, a probabilistic CBF layer evaluates and modifies unsafe high-level sub-goal assignments, preventing unsafe coordination decisions from propagating to the motion controller. Each robot then follows its assigned safe sub-goal using a low-level controller or goal-conditioned policy, where a second probabilistic CBF layer filters primitive motion commands under uncertainty. The resulting architecture couples persistent sensing, distributed belief estimation, communication-constrained coordination, and hierarchical safety, enabling robot teams to maintain a fresh estimate of a dynamic field despite partial observability, uncertain hazards, and limited communication.