A Neuro-Symbolic Co-Pilot for Autonomous Driving: Post-Hoc Behavioral Discovery for Hierarchical Decision-Making

Abstract

In the quest for Level 5 autonomous driving, a critical cognitive bottleneck exists between high-frequency mechanical reflexes and low-frequency semantic reasoning. End-to-end Vision-Language-Action (VLA) models provide strong semantic reasoning but introduce inference latency incompatible with real-time vehicle control (often exceeding 1.5s per decision cycle), while purely reactive Reinforcement Learning (RL) controllers preserve sub-millisecond reflexes but remain conceptually opaque. This paper focuses on the decision-making core of a Neuro-Symbolic Co-Pilot built and evaluated in the CARLA simulator: a hierarchical arbitration system that decouples continuous control, discrete behavioral safety, and semantic reasoning into three cooperating layers operating at three different timescales. A Soft Actor-Critic (SAC) agent governs continuous throttle/steer at 50Hz under a six-component, Optuna-tuned reward function. Its central limitation—that a scalar reward cannot reliably encode discrete behavioral safety constraints—motivates our main methodological contribution: post-hoc behavioral abstraction. Rather than hand-labeling driving behaviors, we apply Dynamic Time Warping (DTW) KMeans clustering directly to the trained SAC agent's own trajectory history, extracting five structurally distinct behavioral modes without any human supervision. A Gated Recurrent Unit (GRU), trained on this data-driven taxonomy, then enforces class-conditional brake overrides (90.1% validation accuracy, 0.900 macro-F1) that the SAC's reward signal cannot guarantee, while a Small Language Model (Qwen 0.8B) performs a final, asynchronous (2Hz) semantic safety check via max-pooling arbitration. We show that this unsupervised clustering organically discovered a statistically significant kinematic asymmetry between left and right turns (mean speed 10.7 vs. 13.1 km/h; mean obstacle distance 7.9 vs. 28.2 m) that no hand-crafted labeling scheme would have distinguished—direct evidence that a trained policy's own behavioral vocabulary can differ from human intuition, and that safety taxonomies are better extracted from a policy's actual trajectory statistics than imposed a priori. We detail the full reward decomposition, clustering methodology, arbitration logic, and closed-loop results, and compare against recent RL, PPO-shielded, and LLM-driven decision architectures in CARLA.