Ensemble Voting of Large Language Models for Multiple-Choice Answer Prediction

Abstract

Machine Reading Comprehension (MRC) has become a key benchmark for evaluating natural language processing systems, and multiple-choice question answering is widely adopted for assessing reasoning, comprehension, and commonsense inference capabilities. However, traditional embedding-based approaches often struggle with complex reasoning and have limited access to external knowledge. Although Large Language Models (LLMs) have demonstrated strong language understanding and reasoning abilities, relying on a single model remains challenging because of hallucinations, model biases, and differences in training data, architectures, and domain expertise. Consequently, no single model consistently performs best across diverse multiple-choice question answering tasks. To address these limitations, we propose an ensemble framework for multiple-choice question answering. The framework incorporates a model selection mechanism that identifies model combinations with complementary error patterns to improve the error-correction capability of the ensemble. In addition, when disagreements occur among the selected base models, a Thinking Model is dynamically invoked to resolve conflicting predictions, thereby reserving advanced reasoning for cases requiring additional resolution. We evaluate nine open-source LLMs on five datasets in Chinese and English. Each dataset is partitioned into 30% validation data for model selection and 70% held-out test data for final evaluation, with the experiment independently repeated five times. Experimental results show that the proposed framework outperforms the individual LLMs on five datasets and achieves an average error-rate reduction ranging from 3.46% to 7.14% relative to conventional ensemble baselines. Furthermore, the proposed approach reduces the average number of Thinking Model invocations by 87.83% compared with the strongest benchmark model while maintaining comparable accuracy, demonstrating a favorable accuracy–efficiency trade-off.