Speech-guided unmanned aerial vehicle control system based on large language model-structured execution graphs with expert rule constraints

Abstract

As the use of Unmanned Aerial Vehicles (UAVs) expands into industrial and safety-critical applications, there is an increasing need for intuitive speech-driven control systems that reduce reliance on predefined command sets or manual interfaces. Natural language instructions, however, often exhibit ambiguity, logical inconsistency, and a lack of structured representation, which limits their direct applicability to UAV operations. This research develops a speech-guided UAV control system in which large language models (LLMs) convert natural-language instructions into typed execution graphs and explicit expert knowledge constrains them before physical execution. This neuro-symbolic design separates probabilistic language interpretation from deterministic execution authorization while retaining in-context prompting for flexible command understanding without parameter fine-tuning. The system was evaluated through command comprehension, graph structuring, validation, robustness testing, user interaction, and indoor physical execution. It achieved up to 95.4% command-comprehension accuracy and 89.6% overall graph-structuring accuracy, outperforming non-large-language-model baselines that remained below 76.0%. The expert-rule validator detected all predefined violation types in 2100 faulty graphs with 0.0% false rejection on 1000 valid graphs. Pilot and indoor physical tests achieved 88.3% user-task success and 93.8% physical execution success among validated commands. These results demonstrate a practical and interpretable framework for task-level UAV command interpretation and mission planning, with dedicated low-level flight control handling reactive stabilization.