Unsupervised Annotation Transfer in Phase-Contrast Microscopy Using a CycleGAN

Abstract

Domain shift between microscopy imaging domains poses a significant challenge for deploying deep learning-based cell detection models across different experimental setups. Manual annotation of new microscopy datasets remains resource-intensive and time-consuming. This study presents a detection-oriented Cycle-Consistent Generative Adversarial Network (CycleGAN)-based annotation transfer framework for adapting a labeled B16BL6 source domain to a HeLa target domain. Annotated B16BL6 melanoma microscopy images are translated into the visual appearance of HeLa microscopy data while retaining their original bounding-box annotations, enabling YOLOv8x detector training without full manual annotation of the target domain. Three YOLOv8x configurations were compared: a source-only bright B16BL6 baseline, an intensity-inverted dark B16BL6 baseline, and the proposed CycleGAN-translated B16BL6 → HeLa configuration. Performance was evaluated using standard object detection metrics on a manually annotated 100-frame HeLa target-domain subset, together with complementary unsupervised proxy metrics on the full unlabeled HeLa dataset. The CycleGAN-trained detector achieved the highest supervised target-domain performance, with a precision of 0.244, recall of 0.353, F1-score of 0.288, and mAP@0.50 of 0.186, compared with mAP@0.50 values of 0.027 and 0.009 for the bright and dark baselines, respectively. It also achieved the highest exploratory composite reliability score on the full HeLa sequence. These findings demonstrate that source-to-target image translation improves detector generalization under the investigated B16BL6 → HeLa phase-contrast microscopy domain shift, thereby reducing the need for extensive manual target-domain annotation.