How to train an LLM for multi-task correction?
NEKO: Toward Post Recognition Generative Correction Large Language Models with Task-Oriented Experts
This paper introduces NEKO, a multi-task generative error correction model for post-processing outputs from ASR, MT, ST, OCR, and TEC systems. It leverages a Mixture-of-Experts (MoE) architecture, where each expert specializes in a specific task during training.
For LLM-based multi-agent systems, the key takeaway is the use of MoE to manage distinct expert agents, each trained on a different modality or task. This specialization allows knowledge sharing between agents through the router while maintaining task-specific expertise, potentially creating a more robust and generalizable system compared to training a single massive model for all tasks. This work demonstrates the viability of using task-oriented MoE training with LLMs in a multi-agent context for specialized output correction across several application areas, and it achieves state-of-the-art results on standard benchmarks such as ASR and speech translation tasks.