How can I model complex agent beliefs about rationality?
Uncommon Belief in Rationality
This paper proposes a graph-based model called RBR (Rationality and Beliefs in Rationality) graphs to represent complex higher-order beliefs about rationality in multi-agent systems, moving beyond the simpler common knowledge/belief assumptions. It introduces "doxastic rationalizability" as a solution concept for predicting agent behavior in games with uncommon belief structures and an algorithm to compress these graphs into minimal forms for efficient computation.
For LLM-based multi-agent systems, the key takeaway is the ability to model nuanced rationality assumptions using RBR graphs, allowing for more realistic and complex agent interactions. The efficient graph compression algorithm could prove useful for managing the computational complexity associated with higher-order beliefs in practical implementations involving LLMs. The iterative rationalization process, driven by the graph structure, offers a mechanism to simulate LLM agents reasoning about each other's actions and beliefs in dynamic contexts.