How can agents share surprise for better adaptation?
Communicating Unexpectedness for Out-of-Distribution Multi-Agent Reinforcement Learning
This paper introduces Unexpected Encoding Scheme with Reward (UES+R), a communication method for decentralized multi-agent reinforcement learning (MARL) designed to improve robustness in unexpected situations. Agents communicate not only reward-related information, but also "unexpectedness"—the difference between predicted and actual observations. This helps agents adapt to changes in the environment not seen during training.
For LLM-based multi-agent systems, UES+R suggests a valuable approach for inter-agent communication. Sharing "unexpectedness" could enable LLMs to better coordinate and learn from surprising situations, potentially leading to more robust and adaptable multi-agent applications. The concept of encoding and sharing discrepancies between expected and actual outcomes is directly applicable to scenarios where LLMs interact with dynamic environments and each other.