Effects of SO(3) Action Representations on Observation-based Imitation Learning for Robotic Manipulation

Abstract

This study examines how 3D rotation action representations affect observation-based imitation learning for robotic manipulation. In behavioral cloning from observation (BCO), expert actions are unavailable, so an inverse dynamics model (IDM) estimates pseudo-actions from consecutive observations that supervise the policy. To identify suitable rotation action representations for imitation learning, we propose a unified BC/BCO framework that preserves the robosuite-compatible 7D action interface while varying only the internal rotation representation. Axis-angle, quaternion, and 6D rotation representations are evaluated with multilayer perceptron (MLP) and recurrent neural network (RNN) policies on the robomimic Lift, Can, and NutAssemblySquare tasks under a common protocol. Given the mean-squared-error losses in each representation space, quaternion or 6D achieves the highest mean success rate in 11 of 12 task–method combinations. On Square, 6D performs best for BC-RNN and attains the same performance as the quaternion for BCO-RNN. In BCO, IDM pseudo-action quality is comparable across representations on Square. These results suggest that rotation action representation matters in imitation learning, although the best choice depends on the task and policy structure.