Can shared knowledge improve LLM agent accuracy?
AKIBoards: A Structure-Following Multiagent System for Predicting Acute Kidney Injury
April 30, 2025
https://arxiv.org/pdf/2504.20368This paper introduces STRUC-MAS (STRUCture-following for Multiagent Systems), a framework for building multi-agent systems that learn and follow a shared "global structure" of a problem. It demonstrates this framework with AKIBoards, a multi-agent system for predicting Acute Kidney Injury (AKI).
Key points for LLM-based multi-agent systems:
- Global structure improves performance: Learning and incorporating a global structure of the data (relationships between variables) significantly improved the performance of individual LLMs in predicting AKI compared to LLMs without this shared knowledge.
- Agent interaction refines beliefs: Explicit interactions between agents, simulating a "smart rounds" scenario, allowed agents to refine their predictions and confidence levels, akin to knowledge distillation. Weaker agents learned from stronger ones, and stronger agents reinforced their beliefs.
- Explainability is key: The system uses agent-based terms (ABT) and multi-agent records (MAR) to log and analyze agent interactions and reasoning, providing insights into their decision-making processes.
- Prosocial focus: The framework incorporates a "prosocial layer" to ensure responsible use, aiming to augment rather than replace human expertise.
- RAG augments structure-following: Retrieval Augmented Generation (RAG) further enhanced performance when combined with structure-following, allowing agents to leverage similar past cases.
- Smart rounds for efficiency: The "smart rounds" approach optimizes agent interactions to reach consensus efficiently, employing early stopping when performance gains plateau.