How to teach AI agents safe interaction?
Learning responsibility allocations for multi-agent interactions: A differentiable optimization approach with control barrier functions
This paper introduces a novel method for understanding and quantifying "responsibility" in multi-agent interactions, particularly focusing on collision avoidance. It leverages Control Barrier Functions (CBFs) within a differentiable optimization framework to learn how much each agent should deviate from its desired trajectory to ensure safety, effectively capturing implicit social norms from data.
This approach is relevant to LLM-based multi-agent systems as it provides a data-driven way to: (1) analyze and interpret the behavior of multi-agent systems trained on real-world data, and (2) potentially guide the design of socially-aware LLM agents by incorporating learned responsibility allocations into their decision-making processes.