AI maps of the 3D world are advancing, but key tests are still missing
A review finds that 3D scene graphs can connect geometry, objects and their relationships, while evaluation remains fragmented.
Editorial illustration — not from the study.
The review brings together research on how 3D scene graphs are designed, built from sensor data, used in robotics and computer vision, and evaluated. It describes a shift from compact descriptions of static scenes toward online representations that can handle changing environments, open-ended language and tasks such as navigation, manipulation and planning.
The authors also identify major gaps. Studies often assess the quality of a graph separately from the performance of the task that uses it, making it difficult to tell how much the representation contributes. The review points to task-specific detail, compact reasoning systems, combined explicit and implicit scene models, and persistent models that predict change as future research directions.
What the review finds
The authors propose a common framework for 3D scene graphs, which represent a three-dimensional environment through nodes and connections that can capture geometry, semantic information, hierarchy, changing states and possible actions. They organize existing work around representation, construction, applications and evaluation, helping distinguish terms and approaches used by different research communities.
Across the literature, the review finds that the field has moved from processing reconstructed, static scenes in batches toward incremental systems that update representations as new observations arrive. It also finds growing use of language and vision-language systems as interfaces for navigation, manipulation, planning and question answering.
The review identifies evaluation as the most widespread unresolved problem. Graph quality and task performance are commonly measured independently, often with different data, so the effect of perception errors on later behavior is rarely measured directly. Other gaps include weak support for spatial consistency in many learning-based systems, limited integration of indoor and outdoor settings, and relatively little treatment of time, scene history and changes caused by interactions.
Why shared 3D maps matter
A representation that links precise locations with objects, relationships and possible actions could give robots and other spatial AI systems a more usable account of their surroundings. The review suggests that this common structure may help connect perception with language, memory, planning and physical action.
At the same time, the findings show why progress cannot be judged only by whether a system produces a plausible scene graph. For real-world use, researchers need to determine whether errors in the graph change what a robot does, and whether representations can remain coherent as environments change. More consistent testing would make comparisons between systems and claims about practical usefulness clearer.
Evidence and open limits
This is a survey and critical review of existing research, not a new experiment or a direct test of one robot system. Its conclusions depend on the methods, datasets and evaluation practices reported in the literature it examines. The authors note that the field is fragmented, with different definitions, construction pipelines and metrics, and that the review focuses on representative papers while providing a broader searchable publication database through its companion website.
// Source
Annual Review of Control Robotics and Autonomous Systems · 2026 · DOI: 10.1146/annurev-control-031025-014243
Researchers developed a model for predicting how molecules alter the electrical current as they pass through nanopores. The accompanying web app estimates signals for different molecule shapes, pore designs and positions, including molecules near the pore opening.
Researchers developed a model that predicts how branching cables deform while a robot moves them. Tested on six physical cable assemblies, it made predictions at more than 100 updates per second and supported several manipulation tasks.
A modeling framework traced a Bluetooth 5.0 prototype’s 3.2-decibel receiver sensitivity shortfall to a mismatch between its antenna and amplifier. After two matching-network adjustments, sensitivity returned to within 0.1 decibels of the design target.