UC Food Waste: A Multimodal Robotic Perception Dataset for Robust Contaminant Detection in Complex Biowaste Scenes

Abstract

Automated organic waste recovery represents a significant challenge for computer vision, characterized by extreme spatial clutter and the material degradation inherent in industrial waste streams. While multimodal imaging provides a potential pathway for material identification, standard deep learning frameworks often prioritize perceptual image enhancement over the preservation of the discrete physical signatures required for accurate classification. This article introduces the UC Food Waste dataset, a comprehensive multimodal resource comprising 15‐band multispectral, thermal, and RGB imagery, designed to characterize contaminants in messy sorting environments. To address limitations of conventional sensor integration, we propose a Split Spatial‐Spectral Attention stem. This architecture utilizes 3D convolutional kernels to explicitly model interband correlations before cross‐modal feature integration, preserving the integrity of spectral signatures that are often suppressed by 2D‐only architectures. Our experiments demonstrate that while feature‐level fusion consistently outperforms data‐level concatenation, the efficacy of multimodal integration in practical settings is heavily constrained by label ambiguity and class overlap. The results suggest that for domain‐specific robotic tasks, the explicit codesign of sensor modalities and feature‐extraction layers are effective for identifying waste contaminants. This, in turn, suggests that in industrial settings extra constraints such as spatial ambiguities must be accounted for in the modeling process.