A Survey on Parallel Reasoning

Abstract

Abstract As Large Language Models (LLMs) evolve, parallel reasoning has emerged as a vital inference paradigm that enhances robustness by concurrently exploring multiple thought trajectories. Unlike fragile sequential methods, parallel reasoning expands inference breadth to significantly improve problem-solving performance. This paper provides a comprehensive survey of the progress and challenges in this burgeoning field. We first formally define parallel reasoning and distinguish it from sequential paradigms like Chain-of-Thought. Then, we propose a novel taxonomy to categorize advanced techniques into non-interactive reasoning, interactive collaboration, and efficiency-oriented decoding strategies. Furthermore, we examine diverse application scenarios, including complex problem-solving and reliability enhancement. Finally, we identify core challenges and outline future research directions. This work serves as a strategic roadmap to foster further innovation in parallel reasoning.