How does crowd opinion boost AI performance?
Crowd IQ - Aggregating Opinions to Boost Performance
October 15, 2024
https://arxiv.org/pdf/2410.10004This research explores how combining the "opinions" of multiple individuals on tasks, similar to using multiple LLMs, can lead to better results than relying on a single source, even the "smartest" one.
Key findings relevant to LLM-based multi-agent systems:
- Crowd beats individual: Combining outputs from a group of LLMs, even with simple methods like majority voting, can significantly outperform the best individual LLM.
- Homogeneity matters: This "crowd wisdom" is even more pronounced when the LLMs have similar skill levels.
- Contextual contribution: An LLM's contribution isn't just about its individual capability, but also how its output uniquely complements others, highlighting the importance of diverse LLMs in a multi-agent system.
- Beyond simple aggregation: While simple methods are effective, the paper suggests there's room for more advanced aggregation techniques, potentially leveraging machine learning, to further boost combined performance.