How do governance systems affect agent behavior in simulated economies?
Incentives to Build Houses, Trade Houses, or Trade House Building Skills in Simulated Worlds under Various Governing Systems or Institutions: Comparing Multi-agent Reinforcement Learning to Generative Agent-based Model
November 28, 2024
https://arxiv.org/pdf/2411.17724This paper explores how different governing systems (libertarian, semi-libertarian/utilitarian, and utilitarian) and institutions (inclusive, arbitrary, and extractive) influence agents' economic behavior in a simulated world where agents can build, trade houses, or trade house-building skills. It compares two multi-agent AI approaches: Multi-Agent Reinforcement Learning (MARL) using the AI-Economist framework and Generative Agent-Based Modeling (GABM) using the Concordia framework.
Key points for LLM-based multi-agent systems:
- Comparison of MARL and GABM: The paper offers a qualitative comparison of the strengths and limitations of each approach for simulating socioeconomic phenomena. It investigates how well each approach can model implicit rules of the environment, impacting the agents’ world models and subsequently their planning.
- Agent Behavior under Different Governance: The research analyzes how agents' incentives to build, trade houses, or trade skills shift under different governing systems and institutions within both MARL and GABM simulations.
- LLM Integration in GABM: The Concordia framework uses natural language actions processed by a "game master" (similar to a central planner in MARL) to control the environment and translate agent actions. This showcases a way to integrate LLMs into multi-agent simulations.
- Prompt Engineering and World Modeling: The study highlights the importance of designing prompts and environment descriptions (like those in Figure 3) for the LLM-based agents in Concordia, influencing their understanding and planning within the simulated world. Both MARL and GABM agents demonstrate the ability to infer world models to facilitate planning, despite relying on implicit environmental rules.
- Implicit Rule Learning: The research investigates how effectively MARL and GABM agents can infer a world model from implicitly programmed environment rules and use it for planning. This is important for developing LLM-based agents that can understand and operate in complex, nuanced environments.