VQ-Transformer: A deep learning-driven model for transcription factor sequence identification

Abstract

Background Transcription factors are key proteins that regulate gene expression, and their accurate identification is important for elucidating the mechanisms of life activities. Traditional methods for identifying transcription factor sequences rely on manually designed features (such as sequence alignment) and shallow machine learning models, which suffer from significant information loss and weak modeling capability for long-range dependencies. Methods This paper proposes a dual-branch deep learning model named VQ-Transformer, which innovatively integrates unsupervised learning with the Transformer architecture. First, a Vector Quantized Variational Autoencoder (VQ-VAE) is employed to automatically learn discrete semantic encoding from massive protein sequences, capturing evolutionarily conserved patterns. Second, a convolutional branch is designed to extract local amino acid motif features and dynamically fuse them with the global semantic representations learned by VQ-VAE. Finally, a Transformer encoder is introduced to model long-range dependencies, enabling end-to-end classification. Results Comprehensive comparisons with the existing models on datasets from human, mouse, and Arabidopsis thaliana were conducted. The VQ-Transformer model achieved superior performance across various metrics and multiple species. Conclusion The VQ-Transformer model provides an effective new tool for transcription factor identification, and can be further extended to multi-modal data fusion and synthetic biology component design in the future, supporting the development of functional bioengineering.