Comparative analysis of automatic mask generation methods for agricultural seed instance segmentation

Abstract

In agricultural image analysis, training deep learning models relies on large amounts of pixel-level labeled data. However, individually labeling densely stacked and overlapping objects, such as pea seeds, is both time-consuming and laborious. This study compares five automated mask-generation methods for instance segmentation of pea seeds (Pisum sativum) that do not require domain-specific customization: Cellpose, EfficientSAM, SAM2, StarDist, and FastSAM. A dataset of 164 images was used in the study. 100 of these images were manually annotated by a trained annotator using the LabelMe tool, containing a total of 33,462 instance labels. The remaining 64 images were left unlabeled and evaluated for automated labeling in downstream model training. All methods were evaluated on a fixed testset of 20 images containing 6,849 instances using precision, recall, F1 score, and mean IoU metrics. The results show that Cellpose, developed for biomedical cell segmentation, achieved the highest overall performance on overlapping pea-sized particles (F1: 0.957, Recall: 0.973). EfficientSAM produced the most precise pixel-level masks (mean IoU: 0.970), while SAM2 exhibited high recall (0.965) but low precision (0.815). StarDist struggled with overlapping particles due to shape constraints (mean IoU: 0.866), while FastSAM missed many particles in dense clusters, achieving high precision (0.976) but low recall (0.725). Furthermore, the YOLO11m-seg model, trained on 144 images automatically labeled with Cellpose tags, performed nearly as well as the model trained on only 80 manually labeled images (mAP50: 0.848 vs. 0.860). These findings suggest that automated labeling could be a practical alternative to manual labeling, even in challenging agricultural imagery containing overlapping objects.