Chang Nie

Towards the Vision-Sound-Language-Action paradigm: The HEAR framework for sound-centric manipulation

Humans and animals use sound as a crucial cue for interacting with the physical world, as acoustic events can reveal contact, completion, hidden contents, or process state. Embodied agents should similarly benefit from auditory awareness during manipulation, yet existing Vision-L...