Can LLMs improve MARL credit assignment?
Speaking the Language of Teamwork: LLM-Guided Credit Assignment in Multi-Agent Reinforcement Learning
This paper addresses the credit assignment problem in multi-agent reinforcement learning (MARL), particularly in scenarios with sparse rewards. It proposes using LLMs to generate dense, agent-specific rewards based on natural language task descriptions, guiding agents to learn better collaborative policies.
Key points for LLM-based multi-agent systems: LLMs can effectively decompose sparse team rewards into dense individual rewards. Agent-specific prompts incorporating collaboration dependencies improve credit assignment. Potential-based reward shaping increases robustness to LLM ranking errors, allowing smaller, more accessible LLMs to be used effectively. The proposed method demonstrated faster convergence and higher returns compared to traditional MARL baselines in several test environments.