Exploring Differential Item Functioning through Machine Learning: A Review of Rasch Trees and Regularized Moderated Rasch Models

Abstract

Detecting and interpreting differential item functioning (DIF) is critical for ensuring the fairness and validity of measurement instruments. Combinations of psychometric models with data-driven machine learning (ML) approaches have been introduced to facilitate comprehensive DIF analysis across diverse sets of covariates. We review two ML methods for detecting uniform DIF effects (i.e. variation of item difficulty parameters). Rasch trees explore DIF effects with nonparametric decision trees and divide the dataset into subgroups. Regularized moderated Rasch models specify a parametric model for predicting DIF effects. A simulation study illustrates that the optimization strategy and functional form assumptions can result in different conclusions. Finally, we demonstrate the implementation of the methods in practice, using data ( N = 5,193) from the National Educational Panel Study, and provide documented analysis code.