Can AI coach human teams effectively?
SOCRATIC: Enhancing Human Teamwork via AI-enabled Coaching
February 26, 2025
https://arxiv.org/pdf/2502.17643This paper introduces SOCRATIC, an AI system designed to enhance human teamwork in real-time, particularly for complex, time-sensitive tasks like disaster response or surgery. SOCRATIC acts as a virtual coach, monitoring team actions, inferring team members' intentions, and offering targeted suggestions when it detects potential misalignments that could lead to suboptimal performance.
Key points for LLM-based multi-agent systems:
- Intent inference: SOCRATIC uses a Bayesian approach and leverages multi-agent imitation learning (specifically BTIL) trained on human demonstrations to model team behavior and infer the likely intentions of each team member, even with partial information. This is relevant to building LLMs that can understand and predict the goals and actions of multiple agents in a collaborative setting.
- Intervention generation: The system uses a cost-benefit analysis based on the inferred intents to decide when and how to intervene, aiming to provide helpful guidance without disrupting the team's workflow. This is analogous to prompting strategies in LLMs, where the system needs to decide when to provide feedback or suggestions.
- Human-AI collaboration: SOCRATIC is designed to work with human teams, not replace them. It focuses on improving coordination and communication among human members by providing real-time support and feedback. This aligns with the growing focus on building LLMs that can collaborate effectively with humans in various tasks.
- Task-time coaching: The focus is on delivering real-time, in-the-moment feedback during task execution, as opposed to post-hoc analysis. This is relevant to building interactive LLM applications where the system needs to provide dynamic and context-aware responses.
- User interface: SOCRATIC uses a user-friendly interface to deliver its interventions to the human team. This highlights the importance of designing intuitive interfaces for LLM-based systems to facilitate seamless human-AI interaction.