Active Visual Perception: Opportunities and Challenges

Abstract

Active visual perception enables an intelligent system to improve perception by controlling how visual evidence is acquired. However, differences in how studies define active perception and evaluate its benefits make approaches difficult to compare across tasks. This review first clarifies the field’s functional scope by examining whether an action is selected for its expected effect on subsequent visual evidence and whether the resulting observation informs a later decision. This definition distinguishes active observation acquisition from processing fixed inputs and guides the selection of representative literature. Within this scope, this review develops an analytical framework organized around four questions. It begins with what evidence the task requires, examines which actions can obtain that evidence and how a candidate observation should be valued, and considers how the next sensing action should be selected and executed. New observations then update the perceptual state and guide subsequent decisions, forming a closed loop that provides a common basis for comparing methods across applications. To examine how their benefits can be demonstrated, this review assesses whether existing datasets support observation selection, how interactive environments enable controlled comparisons, and what physical platforms reveal about execution feasibility. It further examines how baseline comparisons can establish whether adapting observations is necessary and whether informed selection offers benefits beyond simpler sensing strategies. These considerations underpin an evaluation framework that links improvements in visual evidence to downstream task outcomes while accounting for acquisition costs and risks. Its core value is to help isolate the contribution of sensing decisions and determine whether their benefits justify the resources required. The analysis concludes that predicted observation value does not guarantee that useful evidence can be acquired through feasible actions, while task success alone cannot establish the benefit of active observation selection. Further progress requires reliable value estimation and planning over longer decision sequences under physical and resource constraints. It also requires grounding semantic reasoning in acquired evidence and maintaining effective collaboration under imperfect communication, with these capabilities verified through reproducible evaluation of the complete perception and action loop.