HAKEM: Multilingual Spoken-Item Selection Without Transcription

Abstract

Research preprint; not peer reviewed. HAKEM studies multilingual spoken-item selection from a supplied typed catalogue using frozen speech models. It scores each candidate under teacher forcing and subtracts its matched digital-silence score, without task-specific weight training, candidate speech enrolment or a pronunciation lexicon. The main read-aloud Hassaniya study evaluates 2,892 recordings from 16 speakers against 1,491 catalogue rows. Frozen Whisper tiny achieves 96.6% top-1 selection, compared with 65.6% for transcription followed by catalogue matching. The paper also reports a 22-language extension, simulated channel/noise experiments, a 400-clip English phrase-selection comparison, and an exploratory 853-clip low-resource rewarded-beam extension. The results concern closed-catalogue selection with the correct item supplied. They do not establish full speech-transcription accuracy, spontaneous ordered-command understanding, universal language coverage or world-best performance. The manuscript discloses protocol amendments, failed original criteria, generation failures and reproduction limitations. Hassaniya recordings will remain private and will not be distributed with the paper. Private catalogue rows and identities are not redistributed; other data can be shared only under their applicable licences and release rights. The author confirms speaker agreement to research use and publication of findings while Hassaniya recordings remain private. Some main-study checkpoint and fine-tune lineage remain unresolved. Internal manuscript version: Draft 6.