Surface-Token-Guided Query Refinement for Weakly Supervised RGB Apple Defect Sensing and Visible Surface-Condition Scoring

Abstract

Low-cost RGB imaging is attractive for postharvest apple grading, yet an image-level defect label does not reveal where damage occurs or how much of the visible surface it affects. This study examines whether folder-labeled images can support defect recognition, weak localization, binary health assessment, and surface-condition scoring within one model. The resulting framework, WS-DA-DETR, retains DETR set prediction but conditions its queries on local texture residuals, boundaries, and color deviations extracted from the apple surface. Appearance-derived masks and boxes provide a conservative localization curriculum, while auxiliary contrastive and cross-head constraints relate spatial predictions to image-level condition. Manual spatial annotations are reserved for held-out evaluation. Experiments used 10,000 real and synthetic apple images in exact-duplicate-grouped mixed-domain and cross-domain protocols. The mixed-domain test yielded 89.5% four-class accuracy and 96.3% health AUC. On a held-out manually annotated subset, AP50 was 54.6%, AP50:95 was 32.3%, mask IoU was 48.7%, and localization recall at IoU 0.5 was 74.5%. The Pearson and Spearman correlations with human surface-condition ratings were 0.68 and 0.66, respectively. Model-only inference required 30.2 ms per image (33.1 FPS) on an RTX 3090 GPU (NVIDIA Corporation, Santa Clara, CA, USA). These results support a proof of concept on centered, single-apple images, using folder labels and appearance-derived spatial targets for training and a separate 200-image manual subset for evaluation.