SwinFood–MOE fine-grained food recognition based on hybrid expert architecture

Abstract

With the development of deep learning, food image classification has become of great significance in the food industry and health management. However, high heterogeneity, diversity, complex background, and fine-grained features make recognition challenging. This paper proposes the SwinFood–MOE model based on Swin Transformer, two-branch convolution, and mixture of experts (MOE): data augmentation is used to improve generalization ability, Swin Transformer is used to extract features, and two-branch convolution is used to enhance representation. The Swin Transformer two-stage, four-stage, and two-branch convolution features are pyramid pooled, and four feedforward expert networks and routers are introduced. After automatic weighted aggregation, fully connected classification is used to achieve fine-grained recognition. Experiments on the Food-101 and UECFood-256 datasets show that the accuracy rates reach 99.61% and 94.65%, respectively, significantly improving recognition performance.