Robust recognition of Tamil inscription letters using a hybrid model combining recurrent neural networks, graph neural networks, and attention mechanisms

Abstract

Background Character recognition in ancient Tamil inscriptions is a difficult process because of the complex nature, overlapping, and varied sizes of the characters. Traditional Optical Character Recognition (OCR) methods cannot perform optimally in such situations, making it important to develop an efficient character recognition system. Methodology The proposed system is named CUGRA, which represents the integration of five models, namely, Canny, U-Net, Graph Neural Networks (GNN), Recurrent Neural Networks (RNN), and attention models. Its application for Tamil inscription recognition is referred to as CUGRA-TIR. The process starts with the application of the Canny model for enhancement, followed by the application of the U-Net model for segmentation. A Convolutional Neural Network (CNN) is used for feature extraction, and the features extracted are converted into graph representations by the application of the GNN model, followed by the application of the RNN model with an attention mechanism. The proposed approach mainly focuses on the recognition of characters after preprocessing and segmentation and does not address reconstruction or restoration of the characters. Results On an expanded dataset of ancient Tamil inscriptions, the model was tested against conventional OCR systems and existing deep learning techniques. In five separate trials for model training, a mean accuracy of 96.12% was recorded for model validation, with a median accuracy of 98.98% and an interquartile range of 9.18%. In a separate test set, a total accuracy of 84% was recorded for classification, with a weighted F1-score of 0.83. Conclusion From the results, it is clear that the combined spatial, structural, temporal recognition model capable of reading inscriptions is now much more accurate. The data and code used to make the model are available on GitHub, enabling others to scrutinize them and utilize them to further enhance this work.