Evaluating the classification performance and data requirements of Catch22 features on noisy and short synthetic time series

Abstract

Abstract This study evaluates the performance, data requirements, and computational efficiency of the feature extraction tool Catch22 for time-series classification. Using synthetic sinusoidal signals and nonlinear signals generated with the FitzHugh-Nagumo (FHN) model, we systematically varied signal duration, resolution, noise, class separation and sample size. Six data representations were compared using a support vector machine: raw data, Principal Component Analysis (PCA) applied to raw signals, Fourier features, PCA applied to Fourier features, Catch22 features and PCA applied to Catch22 features. Classification performance was evaluated using the standard metric for balanced datasets—the area under the ROC curve—using repeated stratified cross-validation and non-parametric statistical testing, and p-values used to evaluate statistical significance. Results show that Catch22 features provide good discriminative power even with limited information: In the case of FHN signals simulated with dynamic noise and reduced temporal resolution (3 periods sampled with 7 points per period, i.e., 21 data points), classification is possible when class parameter differences are on the order of 10–20%. The total computation time for feature extraction and classification remains below 50 ms for 200 samples with 21 data points per sample. However, for the signals considered in this study and the classifier employed—a Support Vector Machine with a RBF kernel and tuned parameters— Catch22 does not offer consistently superior classification performance compared to the use of raw data or Fourier-based representations, while requiring substantially longer preprocessing time. Therefore, our results indicate that for the signals considered here, raw or Fourier representations are sufficient for obtaining good classification performance, in spite of the short and noisy nature of the signals.