Lessons learned from a Kaggle challenge for particle picking in cryo-electron tomography

Abstract

Abstract The difficulty of particle picking in cryo-electron tomography remains a barrier to routine in situ structure determination. Machine learning is well-suited to overcome this bottleneck with efficient algorithms that generalize across molecular species. To spur new algorithm development, we held a three-month Kaggle challenge that tasked contestants with annotating five molecular species across hundreds of experimental tomograms. This competition successfully engaged >1000 participants from diverse fields and delivered particle pickers that outperformed existing state-of-the-art. Systematic comparisons of the contestants’ submissions revealed the tolerance of subtomogram averaging to moderate but not severe over-picking and underscored the need for more robust measures of annotation quality. The winning models also highlighted the importance of data augmentation to overcome limited training data. All competition tomograms along with the ground truth and winning teams’ annotations have been released on the CryoET Data Portal as a resource to benchmark current and future tools for particle picking.