How can I design better multi-agent RL tasks needing DOL?
The Composite Task Challenge for Cooperative Multi-Agent Reinforcement Learning
This paper introduces the Composite Task Challenge (CTC), a set of benchmark tasks for cooperative multi-agent reinforcement learning (MARL) designed to specifically test an agent's ability to use division of labor (DOL) and cooperation, unlike existing testbeds. The tasks are structured by combining atomic subtasks with varying levels of information interference, subtask dissimilarity, and subtask quantity. Experiments with established MARL algorithms demonstrate their poor performance on CTC, highlighting the challenge of implementing effective DOL and cooperation. Simplified versions of the tasks show improved performance, confirming solvability while retaining challenge. The research demonstrates a need for more sophisticated MARL algorithms able to handle complex, composite tasks.
The relevance to LLM-based multi-agent systems lies in the highlighted limitations of current MARL methods in scenarios requiring sophisticated cooperation and DOL. This translates directly to the challenges in developing LLM-based agents that can effectively divide complex tasks, share knowledge, and cooperate to achieve a common goal. The CTC framework and its analysis provide valuable insights for designing and evaluating LLM-based multi-agent systems emphasizing collaboration and specialization. The stability analysis is also relevant, as robust, consistent performance is critical for real-world LLM agent deployments.