TalkWithMachines aims to enhance human-robot interaction in interpretable industrial robotics.
The paper explores the integration of Large Language Models (LLMs) and Vision Language Models (VLMs) with robotic perception and control.
This enables robots to understand and execute commands in natural language and perceive their environment through visual and descriptive inputs.
The research focuses on four LLM-assisted simulated robotic control workflows, including low-level control, language-based feedback, visual information usage, and task planning.