The system generated action plans by separating visual input, language, reasoning and movement instead of relying on task-specific demonstrations or policy training.
Researchers developed a robot system that separates seeing, understanding language, reasoning and acting, then connects those functions with numerical signals. A large language model supplies common-sense, mathematical and physical reasoning to produce an action plan through explicit logic.
The system did not use task-specific demonstration data or manipulation-policy training. In six experimental suites covering scene understanding, manipulation, language interaction, complex reasoning and long sequences of actions, it succeeded in 91.67% of 240 real-world trials.
How the robot performed
The researchers presented a robot system that decouples vision, language, reasoning and action. These modules are linked through numerical signals, while a large language model uses common-sense, mathematical and physical reasoning to generate a complete action plan through explicit logic.
The system required neither task-specific demonstration data nor manipulation-policy training. Researchers evaluated it in six experimental suites covering 3D scene understanding, qualitative and quantitative manipulation, language interaction, complex reasoning and long-horizon multi-step autonomy. Across 240 real-world manipulation trials, the robot achieved a 91.67% success rate. The researchers also report that the robot’s behaviors were interpretable within this approach.
// Source
npj Artificial Intelligence · 2026 · DOI: 10.1038/s44387-026-00145-8
Institutions: Hong Kong Polytechnic University, Southeast University