Comparing Human Gaze and Vision–Language Model Saliency Predictions in Safety-Relevant Scenes

Abstract

Human visual attention plays an important role in how people perceive and respond to environments containing potential risks. It is unclear whether vision–language models (VLMs) can predict where people look in such scenes. This exploratory study compares human gaze with VLM saliency predictions in safety-relevant scenes. Ten participants freely viewed 33 scene images while their gaze was recorded with eye-tracking glasses. Four VLMs (GPT-4o, Gemini 2.5 Pro, Gemini 2.5 Flash and Claude Sonnet 4) were prompted to predict gaze points for the same images, and the resulting maps were compared with the human gaze maps using standard saliency metrics. All four models showed positive spatial agreement with human gaze on the standard metrics (mean Pearson correlation 0.53 for GPT-4o), but a simple centre baseline scored higher. After correcting for centre bias, three of the four models remained above chance. The results for GPT-4o were stable across repeated runs and did not depend on the instructions in the prompt. Overall, current VLMs can partly predict where people look in safety-relevant scenes from a prompt alone, without any eye-tracking training data, but their predictions remain limited.