LessMissed: Screenshot-Grounded Element Identity Resolution for Browser-Based Webcam Eye Tracking

Abstract

LessMissed is a browser-based system for resolving which visible web element corresponds to a user's gaze during webcam-based eye tracking. The system combines browser-native element acquisition with screenshot-grounded identity resolution to address a key ambiguity in DOM-based gaze tracking: overlays and composited interface layers can intercept point queries even when the visually intended target lies underneath. A Manifest V3 browser extension records element metadata and gaze-related coordinates, while a screenshot pipeline provides visual context for resolving the intended interface region. The proposed resolver uses DOM hit testing, ancestor traversal, geometric containment, and screenshot-derived regions to identify the most plausible visible target. In a controlled evaluation on a 1440 × 900 viewport, the proposed stack-walking resolver avoided overlay misattribution and agreed with browser-derived ground truth across the tested grid positions. The study demonstrates the feasibility of combining browser semantics with screenshot evidence for more reliable web-element identity resolution, while leaving end-to-end user gaze accuracy as future work.