Vision-based assembly target pose estimation and adaptation for robotic assembly in Modular Integrated Construction

Abstract

Modular Integrated Construction (MiC) accelerates building processes, yet the positioning and alignment of MiC modules on-site remain hazardous and labor-intensive. This paper formulates the robotic perception task for automated MiC module assembly as the pose estimation of assembly targets, which are categorized as either convex or concave structures. A coarse-to-fine pose estimation (C2FPE) method is proposed, where targets are detected and segmented from RGB images for coarse localization, followed by fine-grained pose estimation using RGB-D data. To address the domain gap between laboratory and construction-site environments, a training-free, multi-modal pose adaptation (T2MPA) method is introduced, enabling a pre-trained detection model to be transferred to real-world construction scenarios without manual annotation. Experimental results show that C2FPE achieves Average Distance of model points (ADD) scores of 84.25% and 98.53% on convex and concave targets, respectively, outperforming state-of-the-art methods. T2MPA achieves AP@0.5 scores of 74.19% and 96.00% for convex and concave targets, exceeding existing domain adaptation methods.