How risky is manipulating multi-agent systems?
It's Not All Black and White: Degree of Truthfulness for Risk-Avoiding Agents
This paper introduces "RAT-degree," a metric quantifying a mechanism's robustness to manipulation by risk-avoiding agents. These agents only manipulate if the manipulation is sometimes beneficial and never harmful, often requiring partial knowledge of other agents' actions. A higher RAT-degree means the mechanism is harder to manipulate. The paper analyzes the RAT-degree of several classic mechanisms in different social choice settings (auctions, voting, resource allocation, stable matching) and introduces novel mechanisms with high RAT-degrees. The concept of safe manipulation, quantifying knowledge needed for manipulation, and the focus on risk-averse behavior are directly applicable to the design of robust and predictable LLM-based multi-agent systems. Designing multi-agent interactions with a focus on RAT-degree could make these systems more reliable and resistant to unintended manipulation by individual agents.