Eyegent takes an image and a natural-language query, builds possible interpretations of the relevant object or region, and keeps a probability distribution over them. It then directs simulated fixations toward areas expected to provide the most useful information, updates its beliefs using a vision-language model, and stops when its remaining uncertainty reaches a set risk threshold.

The system uses off-the-shelf components and no gaze supervision or task-specific training. On RefCOCO-Gaze and AiR-D, it performed as well as or better than other zero-shot methods on several measures of semantic, spatial and structural scanpaths, although supervised systems remained ahead on some spatial and timing measures.