Prediction-Based Adaptive Designs for Reducing Wave Nonresponse Rates and Bias in Panel Surveys

Abstract

Machine learning (ML)-based nonresponse prediction in panel surveys enables selective interventions. However, the optimal use of ML predictions in Adaptive Survey Design (ASD) remains uncertain. We propose a method that integrates field experiment results on incentives, questionnaire length, and questionnaire content with ML-based propensity models to simulate ASD strategies ex-post with minimal assumptions. Using German panel data, we show that treating 15 percent of panelists, selected through ML-based predictions and R-indicator analysis, with increased cash incentives or a survey module on the respondents’ preferred topic can reduce nonresponse rates by up to 2 percentage points. These strategies also reduced bias in selected variables. We present our method as a framework for survey researchers to evaluate the expected outcomes of multiple ASD regimes simultaneously in a realistic setting. The framework helps identify both the criteria for allocating treatments to specific participant groups and the most effective modifications to the survey protocol.