How can LLMs learn to adapt to different roles in multi-agent games?
Role Play: Learning Adaptive Role-Specific Strategies in Multi-Agent Interactions
November 5, 2024
https://arxiv.org/pdf/2411.01166This paper introduces Role Play (RP), a novel framework for training adaptable agents in multi-agent reinforcement learning (MARL) scenarios, particularly zero-shot coordination where agents must collaborate with unknown partners. Instead of relying on a pool of diverse policies trained through self-play, RP assigns each agent a "role" represented by an embedding vector, allowing a single policy to generate diverse behaviors. A role predictor helps agents anticipate the behavior of others based on observations and their own roles.
Key points for LLM-based multi-agent systems:
- Role Embeddings: Offers a compact and interpretable way to represent agent personalities and strategies, which could be particularly relevant to LLMs by shaping their response style and goals.
- Role Predictor: Provides a mechanism for agents to infer the intentions and strategies of other agents, aligning with the theory of mind capabilities being explored in LLMs. This could enhance collaboration and strategic decision-making.
- Adaptability: RP aims for improved zero-shot coordination, a crucial aspect for deploying LLMs in dynamic multi-agent environments where they might encounter unseen partners and unexpected situations.
- Simplified Training: Training a single policy with diverse roles is potentially more efficient than managing large policy pools, which could be advantageous for the computationally intensive training of LLM-based agents.