Hyperspectral handwriting analysis for writer verification and ink mismatch detection using a hybrid CNN–LSTM model

Abstract

Hyperspectral imaging captures detailed spectral reflectance information over a broad wavelength range, providing capabilities beyond those of conventional RGB imaging. This rich spectral information can be used in forensic document analysis to identify subtle variations in ink composition and handwriting characteristics that cannot be detected using conventional imaging methods. Nevertheless, current research tends to focus on writer analysis and ink discrimination as two distinct tasks, and in many cases, ignores the potential of their combination.This paper proposes a unified spectral–spatial metric-learning framework for hyperspectral handwriting analysis that jointly addresses writer verification and ink mismatch detection through a shared sentence-level embedding space. Each sample is modeled as a three-dimensional hyperspectral cube containing 149 spectral bands, with spatial dimensions standardized during preprocessing. To minimize variability introduced by image acquisition conditions, band-wise normalization is applied. The framework employs a hybrid CNN–LSTM architecture as the feature learning backbone, where convolutional neural networks extract discriminative spatial handwriting characteristics and the LSTM models spectral dependencies across hyperspectral bands. A contrastive learning strategy optimizes the shared embedding space, enabling similarity-based verification through Euclidean distance between learned representations. Comprehensive experiments conducted on a large-scale hyperspectral handwriting dataset demonstrate the effectiveness of the proposed framework for both tasks. The results demonstrate that learning a unified spectral–spatial representation within a verification-oriented framework improves representation consistency, enhances generalization to previously unseen writers and ink samples, and highlights the practical applicability of hyperspectral imaging for forensic document analysis.