How can I build a decentralized, scalable Gaussian Process ensemble for my multi-agent app?
Decentralized Online Ensembles of Gaussian Processes for Multi-Agent Systems
This paper introduces a decentralized method for multiple agents to collaboratively learn a complex function (like those modeled by LLMs) without a central server. Each agent learns locally and shares information with neighbors to build a shared understanding, similar to a gossip protocol. The method uses a computationally efficient approximation of Gaussian Processes (called Random Fourier Features) which makes it suitable for real-time online learning and scalable to many agents. Further, it employs Bayesian Model Averaging to combine predictions from multiple models with different parameters, enhancing overall prediction accuracy without costly hyperparameter tuning, which is particularly relevant when working with complex LLMs.