Can transformers play games in-context?
Transformers as Game Players: Provable In-context Game-playing Capabilities of Pre-trained Models
October 15, 2024
https://arxiv.org/pdf/2410.09701This paper explores the ability of pre-trained transformer models to learn how to play games in a multi-agent setting without direct training on those specific games (in-context game playing). It focuses on two-player zero-sum games and shows theoretically that transformers can learn approximate Nash Equilibria in both decentralized (separate models for each player) and centralized (single model controlling both players) scenarios.
Key points for LLM-based multi-agent systems:
- Transformers can learn complex game strategies: The paper proves transformers can implement known game-playing algorithms like V-learning (decentralized) and VI-ULCB (centralized).
- Generalization to new games: Pre-trained transformers can adapt to new game environments without parameter updates, suggesting potential for flexible multi-agent systems.
- Decentralized learning is viable: Transformers can learn effective strategies even without observing the opponent's actions, crucial for realistic multi-agent scenarios.
- Theoretical foundation for LLM-based agents: This work lays a theoretical groundwork for building and analyzing LLM-based multi-agent systems.