How to make LLMs use inclusive pronouns?
Mitigating Bias in Queer Representation within Large Language Models: A Collaborative Agent Approach
November 13, 2024
https://arxiv.org/pdf/2411.07656This paper tackles the issue of bias against queer individuals in large language models, specifically focusing on incorrect pronoun usage. It introduces a multi-agent system where different agents collaborate to analyze and correct pronoun usage in LLM-generated text, promoting inclusivity.
Key points for LLM-based multi-agent systems:
- Specialized agents: The system uses specialized agents for bias detection, analysis, and optimization, demonstrating a modular and potentially scalable approach.
- Sequential collaboration: Agents work sequentially, refining each other's output and reducing individual agent bias, showcasing a collaborative multi-agent workflow.
- Transparency and reasoning: Agents provide explanations for their decisions, promoting transparency and user trust, which is crucial for responsible AI development.
- Structured output: JSON schema ensures consistent communication between agents, highlighting the importance of standardized data exchange in multi-agent systems.
- Evaluation on a specialized dataset: The system is tested on the Tango Dataset, demonstrating the importance of benchmark datasets tailored to specific bias types. The results show a significant improvement over GPT-4 in correctly handling gendered and non-binary pronouns.
- Potential for wider application: While focused on pronouns, the multi-agent framework could be adapted to other bias mitigation tasks, suggesting broader applicability in LLM development.