Can removing a player invert tournament rankings?
THE INVERSION PARADOX AND RANKING METHODS IN TOURNAMENTS
This paper explores the "inversion paradox" in ranking systems, where removing the lowest-ranked item can completely reverse the order of the remaining items. It proves that this paradox is unavoidable in any ranking system that satisfies three seemingly reasonable properties. The paradox is demonstrated with several common ranking methods (Borda, Massey, Colley, Markov) and simple tournament structures.
The inversion paradox is highly relevant to multi-agent systems relying on LLMs for ranking. If agents utilize such ranking systems for decision-making (e.g., resource allocation, task prioritization), seemingly inconsequential changes like removing an agent or a task could have drastic, unintended consequences on the overall system behavior. This highlights the need for careful design and robust testing of ranking mechanisms within LLM-based multi-agent applications.