How do network connections affect LLM multi-agent safety?
NetSafe: Exploring the Topological Safety of Multi-agent Network
October 22, 2024
https://arxiv.org/pdf/2410.15686This paper investigates how the structure of a network made up of multiple LLMs (large language models) affects its vulnerability to malicious information, such as misinformation, bias, and harmful content.
Key findings for LLM-based multi-agent systems:
- Connectivity impacts safety: Highly connected LLM networks are more susceptible to attacks, spreading misinformation quickly. Less connected networks are more robust.
- Agent Hallucination: A single LLM's error can spread throughout the network, misleading all the LLMs.
- Aggregation Safety: Despite individual LLM vulnerabilities, multi-agent systems are surprisingly resistant to bias and harmful content due to the collective "safety alignment" of the LLMs.
- Static vs. Dynamic Evaluation: Traditional methods for evaluating network security don't accurately predict real-world performance in these complex systems. Extensive testing is crucial.