How to safely control robots using uncertain predictions?
Safe Decentralized Multi-Agent Control using Black-Box Predictors, Conformal Decision Policies, and Control Barrier Functions
October 1, 2024
https://arxiv.org/pdf/2409.18862This paper proposes a new method for controlling autonomous agents in multi-agent environments where agents rely on potentially inaccurate predictions of each other's movements (like those from LLMs). It combines:
- Control Barrier Functions (CBFs): These are mathematical functions used to ensure safety, like avoiding collisions.
- Conformal Decision Theory (CDT): This framework helps make decisions under uncertainty by adjusting actions based on observed prediction errors.
The key idea is to adapt the restrictiveness of CBF-based safety constraints based on how accurate the predictions are. This allows the agent to balance safety with performance. The system is designed to work even when predictions are inaccurate, providing a mechanism for building more robust LLM-based multi-agent systems.