How best to make decisions in multi-agent LLMs?
Voting or Consensus? Decision-Making in Multi-Agent Debate
This paper investigates how different decision-making protocols (consensus vs. voting) impact the performance of multi-agent LLM systems in solving knowledge-based and reasoning-based tasks. It finds voting is more effective for reasoning, while consensus is better for knowledge tasks. Two novel methods are introduced: All-Agents Drafting (AAD) and Collective Improvement (CI), which boost performance by encouraging answer diversity. Increasing the number of agents is shown to be more beneficial than increasing discussion rounds. The research highlights the importance of tailoring decision protocols to specific task types in multi-agent LLM applications and leveraging answer diversity for optimal performance.