A new dataset shows that local AI processing can detect people and vehicles without relying on continuous video transmission.
The Constellation dataset contains human-labeled images from a high-mounted camera overlooking a dense New York City intersection. The images cover different times of day, seasons, weather conditions and changing backgrounds, including scenes with crowds and small, difficult-to-see pedestrians.
The researchers tested several object-detection systems and evaluated whether they could run on devices with limited computing resources. Their best-performing system reached 92.0% average precision for pedestrians and 95.4% overall average precision on an A100 computer. A model running on a Jetson Orin Nano reached 94.5% mean average precision with 27.5 milliseconds of processing time using TensorRT.
What the cameras detected
Small pedestrians were harder for the tested systems to detect than vehicles, with average precision about 10% lower. Pretraining on structurally similar data improved mean average precision by 1.8%, while domain-specific image changes and machine-generated labels from strong models provided additional gains.
The best computer-based model reached 92.0% pedestrian average precision at an intersection-level threshold and 95.4% overall mean average precision, with 7.08 milliseconds of inference time on an A100 machine. On the edge, the best reported result was 94.5% mean average precision on a Jetson Orin Nano, with 27.5 milliseconds of inference time using TensorRT. The researchers also found performance drift when models trained on data from one time period were compared across changing intersection conditions.
Why local processing matters
The results support the feasibility of analyzing high-mounted intersection-camera footage directly on nearby hardware rather than sending continuous video to a central system. Local processing could reduce data transmission and centralized computing while providing low-latency results for applications such as safety warnings and traffic analysis.
The work also identifies a practical obstacle: people viewed from above can occupy very few pixels, and changing shadows, lighting, weather and backgrounds can reduce detection performance. That gap matters for any future intersection-monitoring system in which detecting pedestrians is important.
Evidence and limits
This is an evaluation of object-detection systems on a newly released dataset, not a test of safety outcomes in live traffic operations. Constellation contains 13,314 annotated images from 28 time intervals, collected by one high-altitude camera at a dense New York City intersection during 2019–2020 and 2023. The dataset includes varied lighting, seasons, weather and backgrounds, but the excerpts do not establish how well the results would transfer to other intersections, camera positions or cities.
The study shows that on-device processing is technically feasible, but it also reports a continuing performance gap between pedestrian and vehicle detection and performance drift across changing conditions. Those limitations remain relevant before such systems can be widely used for safety-critical monitoring.
