How to explain AI agent action impact in multi-agent scenarios?
COUNTERFACTUAL EFFECT DECOMPOSITION IN MULTI-AGENT SEQUENTIAL DECISION MAKING
October 17, 2024
https://arxiv.org/pdf/2410.12539This paper proposes a new method to explain the effects of an AI agent's actions in a multi-agent system. It breaks down the impact of an action into two parts: how other agents would respond, and how the environment itself would change.
The method is particularly relevant to LLM-based multi-agent systems as it can handle complex scenarios with multiple agents instructed by LLMs. It offers a fine-grained analysis of how individual agents and environmental factors contribute to an outcome, exceeding simple blame attribution. This is crucial for understanding and debugging LLM-driven multi-agent interactions.