Evaluation Bias in Cross-Context Generalization of EEG-Based Affective Computing: A Simulation Study

Abstract

Cross-context accuracy in electroencephalography (EEG)-based emotion recognition is usually reported without the model-selection rule, chance-level test, or reliability reference needed to interpret it. We quantify the consequences in simulations with known ground truth and on real EEG features with randomly assigned labels. Selecting a linear classifier’s regularization parameter on the test set gave mean cross-context accuracies of 0.706 (two classes; chance 0.500) and 0.212 (seven classes; chance 0.143) on data without signal, while nested cross-validation stayed at chance. Real EEG features from 89 participants gave 0.640 and 0.207. Because the offset is shared by all participants, group-level t-tests were falsely significant in 95–100% of groups of 10–60 participants. The offset was nearly constant across true effect sizes and appeared for every model class tested, for pooled feature selection, and for repeated test-set reuse. Representational similarity analysis was less sensitive than decoding when conditions shared mean patterns across contexts, and it alone detected structure that was shared relationally but carried by different features. A single condition carrying a power artifact produced a spurious cross-context geometry. An exact permutation test was falsely significant in about one-third of realizations, whereas a conditional permutation test and condition dropping remained calibrated. We derive five reporting items from these results.