How well do LLMs persuade in games?
Among Them: A game-based framework for assessing persuasion capabilities of LLMs
This paper introduces "Among Them," a game-based framework inspired by Among Us, to evaluate the persuasion and deception capabilities of LLMs in a multi-agent setting. Key points for LLM-based multi-agent systems: LLMs can effectively use various persuasion techniques without explicit training, larger model size doesn't necessarily mean better persuasion, and excessive output can be detrimental. The framework analyzes discussions, tagging dialogue with 25 distinct persuasion techniques, providing a detailed view of LLM behavior in social deduction scenarios. The study used eight different LLMs, including variations in size and type, in a tournament setting, revealing nuanced differences in their persuasive strategies. All data and code are open-sourced for further research.