Can LLMs better simulate group decisions?
MF-LLM: Simulating Collective Decision Dynamics via a Mean-Field Large Language Model Framework
May 1, 2025
https://arxiv.org/pdf/2504.21582This paper introduces MF-LLM, a framework using Large Language Models (LLMs) to simulate collective decision-making in groups of AI agents. It addresses the challenge of realistically simulating how individual AI agents' choices influence and are influenced by the overall group's behavior over time.
Key points for LLM-based multi-agent systems include:
- Mean-Field Modeling: Simplifies agent interactions by having agents respond to an aggregated "mean-field" representation of the group, rather than tracking every individual interaction. This makes large-scale simulations feasible.
- Two-Module Architecture: Employs a policy model LLM to generate individual actions based on agent state and the mean-field, and a mean-field model LLM to update the mean-field based on the latest actions. This creates a feedback loop between micro (individual) and macro (group) levels.
- IB-Tune Algorithm: A novel fine-tuning method based on the information bottleneck principle. It optimizes both LLMs jointly: the mean-field model to capture relevant population-level signals for influencing future decisions, and the policy model to generate realistic actions given those signals. This enhances long-horizon decision accuracy.
- Exogenous Signals: Demonstrates how incorporating external events or information into the simulation improves fidelity to real-world dynamics. This allows modeling of disruptions or planned interventions that influence the group.
- Scalability and Generalizability: The framework scales to large agent populations and generalizes across different domains and LLM backbones, even without task-specific tuning.
- Smaller LLMs May Be Better: Experiments suggest smaller LLMs can sometimes outperform larger models in simulating diverse population behavior due to their greater sensitivity to variations in agent state, avoiding the problem of homogeneous responses.