How to train agents in large populations with limited rationality?
Bounded Rationality Equilibrium Learning in Mean Field Games
This paper explores bounded rationality in Mean Field Games (MFGs), a framework for modeling large agent populations. It introduces Quantal Response Equilibria (QRE) and Receding Horizon (RH) MFGs, where agents have noisy reward perceptions and limited planning horizons, respectively. These concepts offer more realistic agent behavior compared to perfect rationality assumptions of Nash Equilibria. Generalized Fixed Point Iteration and Fictitious Play algorithms are adapted to learn these new equilibria.
For LLM-based multi-agent systems, QRE offers a way to model agents with imperfect understanding derived from noisy LLM outputs. RH-MFGs address limited context windows and computational constraints by focusing on short-term planning, similar to how LLMs operate with token limits. The adapted learning algorithms provide practical tools for training and deploying such systems.