Genetic algorithm based automated test assembly for mathematical literacy assessment inspired by the PISA framework

Abstract

Automated test assembly can reduce the manual burden of test construction while preserving predefined content and difficulty constraints. This study develops a genetic algorithm (GA) based automated test assembly (ATA) method for mathematical literacy assessment. The competency-oriented blueprint is inspired by the PISA mathematics framework, whereas the assembly logic uses Classical Test Theory (CTT) item parameters rather than reproducing the official PISA psychometric design. A 120-item bank was encoded as binary chromosomes, and a population of 500 candidate forms was optimized under a 25-item length constraint using a fitness function that combines content-domain deviation (alpha = 0.4) and target-difficulty deviation (beta=0.6). Empirical validation used 42 valid student response records. The assembled test achieved a weighted empirical difficulty of 0.639 against the target value of 0.650, an absolute deviation of 0.011. Eighteen of 25 items (72%) were consistent with their preset difficulty categories. Across the four content domains, the mean absolute deviation between target and achieved score-point percentages was 4.835 percentage points. Item scoring rates were strongly associated with difficulty, with r = − 0.823 ( p < 0.001) for ordinal difficulty codes and r = 0.827 ( p < 0.001) for preset expected scoring rates. These results support the feasibility of GA-based ATA for small-scale mathematical literacy assessment, while the limited sample, CTT-based calibration, moderate item-bank size, and heuristic weight selection constrain generalization.