Can LLMs build robot code from natural language?
AutoMisty: A Multi-Agent LLM Framework for Automated Code Generation in the Misty Social Robot
March 11, 2025
https://arxiv.org/pdf/2503.06791AutoMisty simplifies programming social robots like Misty for non-programmers using natural language instructions. It leverages a multi-agent LLM system where specialized agents handle tasks like planning, actions, touch, and audiovisual processing. A two-layer optimization with self-reflection and human feedback ensures code quality and user alignment. Optimized APIs improve LLM understanding, and memory enhances learning from past interactions. Benchmark tests show AutoMisty outperforms direct prompting of LLMs like ChatGPT in complex robot tasks.