How can we make LLM-MAS reliable?
Position: Towards a Responsible LLM-empowered Multi-Agent Systems
February 5, 2025
https://arxiv.org/pdf/2502.01714This paper discusses the challenges and potential solutions for building responsible and dependable Large Language Model-powered Multi-Agent Systems (LLM-MAS).
Key points for LLM-based multi-agent systems:
- Challenges: Knowledge drift, misinformation propagation, conflicting agreements between agents, LLM hallucinations and potential for collusion, data poisoning and jailbreaking attacks, cybersecurity threats, and the difficulty of evaluating system-level agreement and uncertainty.
- Solutions: Shifting from heuristic solutions to principled system architectures, integrating uncertainty quantification and management, and incorporating human-centered dynamic moderation. Specific methods include probabilistic frameworks, formal verification, belief-desire-intention architectures with conflict resolution, runtime monitoring, AI provenance frameworks, and learning-based evaluation methods. For agent agreement, methods include Reinforcement Learning from Human Feedback (RLHF), Supervised Fine-tuning (SFT), and Self-improvement techniques. For agent-to-agent agreement, methods include Cross-Model Agreement (strong-to-weak and weak-to-strong), Debate and Adversarial Self-Play (Generator-Discriminator and Debate), and Environment Feedback. Uncertainty management focuses on memory retrieval, planning, agent interaction, and robust evaluation techniques involving statistical analysis and human-in-the-loop verification.
- Proposed Framework: A responsible LLM-MAS framework incorporating interdisciplinary perspectives, quantifiable guarantee metrics for agreement and uncertainty, and a moderator integrating symbolic rules with formal verification for dynamic recovery and ensuring system resilience.