Can spatial reasoning improve MARL efficiency?
Investigating Relational State Abstraction in Collaborative MARL
This paper investigates how simplifying state representations based on spatial relationships improves learning efficiency in collaborative multi-agent reinforcement learning (MARL). The researchers introduce MARC (Multi-Agent Relational Critic), an architecture that converts the observed environment into a spatial graph, processed by a relational graph neural network. This approach allows agents to learn from the relative positions of objects and other agents without explicit communication.
For LLM-based multi-agent systems, this research suggests that using relational state abstraction could improve the sample efficiency of LLMs in multi-agent environments. The spatial graph representation might be adaptable to other relational data commonly used by LLMs, offering a potential way to improve training efficiency and generalization in complex, collaborative scenarios. The emphasis on implicit communication through shared representations is also relevant for LLM-based agents, as it can reduce the need for explicit message passing.